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FOREWORD 



Learning-in-work is an integral part of Experienced-Based 
Career Education Programs, internships, cooperative and work- 
experience programs, and on-the-job components of vocational 
educatia^fi. Since the ear ly . 1970 ' s, there has been a movement 
in education to expand the education opportunities of all 
students to include "real world" learning experiences as part 
of the total educational experience. In an attempt to investi- 
gate the relationships of learning and work, the National Center 
for Research in Vocational Education has initiated a programmatic 
effort to conduct basic research of the phenomena. This- study, 
supported by the National Institute of Education, reports the 
findings of an. exploratory examination of student retention of 
mathematical and reading concepts resulting from student enroll- 
ment in a learning-in-work and in a traditional learning 
environment. 

Appreciation is extended to the Anoka-Hennepin School * 
District No. 11, Anoka, Minnesota, for their cooperation and 
participation in the study. Don Anderson, Director of the 
Experience-Based Career Education Program, and Roger Giroux, 
Director of Research and Evaluation, were instrumental in 
providing support for the research staff in their investigation. 

Technical advice for the research effort was provided by 
Harold M. Schroder, Professor and Department Chairman of the 
College of Business Administration at the University of South 
Florida. His scholarly research in the area of complexity 
training and development of individuals was especially useful 
to the research staff. ^ He is also recognized for his critique of 
the report . 

Appreciation is extended to our reviewers for their recom- 
mendations and suggestions: to Joseph Grannis, Professor 
of Education, Teachers College, Columbia University and Stephen 
Franchak, Research Specialist, The National Center for Research 
in Vocational Education. 

Suggestions and extension of the first year's results were 
provided by David P. Ausubel, Distinguished Professor Emeritus, 
Ph.D. Program in Educational Psychology, Graduate School, City 
University of New York; Henry C. Ellis, Chairman, Department 



xi 



of Psychology, University of New Mexico; and Benton J- Undervrood, 
a Stanley G. Harris Professor of Social Science, Northwestern 
University. These scholars were cominissioned to provide a 
perspective for investigating the psychological and pedagogical 
implications of learning and forgetting patterns in learning-in- 
work and traditional environments through extensions of their 
research and learning theories. 

Special appreciation is extended to Ronald Bucknam, National 
Institute of Education Project Officer, for his contributions to 
this report - 

Recognition is due Richard Miguel, Program Director, for the 
support and guidance he provided the study; Michael Crowe/ 
Project Director, for his overall direction of the project and 
the writing of the report; R. J. Harvey, Graduate Research 
Associate, for his assistance in conducting the analyses of the 
data and assisting with the preparation of the report; 
Jeanette McConaughy for her editorial services; and Jackie Masters 
for her secretarial support services. 

Robert E. Taylor 
Executive Director 
The National Center for Research 
in Vocational Education 
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ABSTRACT 



The Study examined the retention of mathematical and reading 
concepts for students enrolled in a learning-in-work environ- 
ment (Experience-Based Career Education) and a traditional 
classroom learning environment on a measure of academic achieve- 
^^^"•^ twelve-month Idrigitudinal design. Student performance 

in each environment (n=27) was evaluated using the Comprehensive 
Tests of Basic Skills, which was administered at the beginning 
and end of their junior year and at the beginning of the senior 
year. Thus , the learning interval was designated as the time 
between pre- and post--testing , and the retention interval, the 
time between post- and follow-up testing. The results indicated 
(1) differences in two areas — in reading skills vs. math skills 
and in traditional learning environments vs. learning -in-work 
environments; (2) math cognitive concepts causally related but 
not in the hierarchical order predicted by Bloom's taxonomy; 

(3) diff.jirent causal processes in the acquisition of math concepts; 
depending upon placement in one of the two learning environments; 

(4) significant relationships between the students V jognitive 
style and their math performance- and (5) moderate relationships 
between the ^^tudents ' perceptions of the complexity of the 
learning environment and their math performance. An interference/ 
assimilation model was proposed to interpret the findings. 
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CHAPTER T 



THE PROBLEM 



Introduction 

This study is predicated on the belief that the learning 
environment affects the satisfaction^ Learning^ and personal 
growth of young people to a considerable degree^ The report 
represents the outcomes of the second year of investigation 
into^ the interaction of students and their: learaing envir-omnient 
and is focused on providing answers to guiestions raised iia the 
preliminary first-year analyses (Crowre and Earveyr IS^S^K The 
study proposes to extend and examine the relationships between 
retention of mathematical concepts and the type of school 
environment in which instruction occurred. These concepts are 
measured by the Comprehensive Tests of Basic Skills (CTBS)) that 
are administered to high school juniors at the beginning and 
close of the school year as well as to seniors at. the-^ beginning 
of their last year in high school. The instructional environments 
under investigation are a traditional school program and a 
learning- in-work program* 



Background 

Learning Environments; Historical Trend^s 

Throughout the last decade, schools, in their attempt to^ , 
'^be more responsive to the employment needs of studeaaits^ were 
affected in the curriculum areas by the Career Education 
movement. This movement emphasized career informaitdloni related 
to student developmental needs, job market trends, correlation, 
between subject matter and the v/orld of work^ and p^lajns fimr 
further education and eventual job selection.. The^ early efforts 
of the career education movement were primarily concerned with? 
restructuring the curriculum used by teachers and s.tudents: in 
the classroom. 



Later efforts^ however/ included the development of coiranunity 
employer-based programs providing students the additional option 
of spending time at the workplace for the purpose of career 
exploration or preparation. In addition to the resurgence of 
vocational and cooperative education ^ new programs such as the 
Experience Based Career Education Program of the National 
Institute of Education ^ the Career Intern Program of the 
Opportunities Industrialization Center / and a privately developed 
program called the Executive Internships Program were developed 
and implemented at the local school level. These programs 
emphasized new arrangements for student learning. An underlying 
belief was that the classroom had to be supplemented (and in 
some cases replaced) by experiences at vrorksites. Although 
students were supervised by certified teachers ^ worksite mentors 
were viewed as credible teachers and role models for students. 

Concurrent developments ^ outside the public educational 
domain, were also taking place with regard to the "hard-core" 
unemployed youth. New CETA legislation/was enacted requiring 
prime sponsors to provide training that would lead to employment 
for targeted groups of youth. Educational Work Councils were 
established by the Department of Labor to coordinate community 
resources for the purpose of training, and placing youth in 
jobs. Organizations such as Youth Work were established to 
develop and pilot programs linking education and work. These 
efforts have led to new paradigms for instructional programs 
and have helped educators / employers, unions, and prime 
sponsors create educational or training programs with 
alternative learning environments to conform more closely with 
educational needs of youth. 

The foregoing suggests that the educational outcomes have 
been expanded to include career exploration, work skills, 
positive attitudes, and personal . growth, while simultaneously 
reinforcing basic skills; that the workplace is an appropriate 
environment for learning some or all of these outcomes; and 
that students are becoming increasingly responsible for selection 
of their educational environment. The cumulative effect of 
legislation and educational practice persuades us that one of 
the major results from these efforts has been legislative 
approval of alternative learning environments for students. 
Generally these environments combine school instructional 
components with involvement at the workplace. Current 
descriptions of programs that involve learning-in-work 
environments can be found in The Current Status of Assessing 
Ex periential Education Programs (Crowe and Adams, 1979) and 



in Experiential Education; A Primer on Programs (Wasson^ 1978). 

Both dociaments reveal the diversity of design tor such 

programs and the increased nximber of youth who are participating. 

Learning Environments; Descriptions 

The basic problem under investigation includes the further 
examination and description of student retention of reading 
and mathematical concepts as a function of their instructional 
or learning environments, in this study a traditional school 
environment and a learning-in-work environments Because 
traditional learning environments are as diverse as 
learning-in-work environments, a comprehensive description is 
not attempted for this study. Nevertheless, our perspective 
of a traditional learning environment includes specific 
structural features ; 

• One teacher for approximately twenty-five students 

• Subject matter organized according to the major 
discipline areas — math, English, sociology 

• Instruction for a subject matter course limited to 
fifty- to sixty-minute intervals (periods) each day 

• Schedule of five to six instructional periods per day 

• Instruction confined to a classroom, under the 
supervision of a teacher who uses curriculum guides 
and/or textbooks as major instructional tools _ 

By way of a general comparison, the learning-in-work 
environments studied for this research includes the following 
structural features : 

• One learning coordinator for approximately twenty-five 
students 

• A resource person (worksite mentor or supervisor) 
identified for each worksite who provides opportunities 
and assistance for students 

• Subject matter organized according to curricular areas, 
thereby cutting across major disciplines (Algebra is 
defined not only as traditional mathematical concepts 
but_also is related to the natural science concepts of 
atomic structure, to electricity, and to the 
English/communications concepts of research and reporting 
skills. ) 

• Course instruction integrated with the worksite exper- 
iences and not confined to fixed intervals of time 



# Rn instructional cycle defined for the student as four 
days a week spent at the worksite and one day at the 
learning center 

• Instruction guided by an activity sheet negotiated by 
the student and the learning coordinator, relating 
subject matter concepts to learning activities and 
providing evaluative criteria for determining 
successful participation in activities 

Students in the tv70 learning environments are exposed to 
different instructional processes and environments related to 
learning^ For ekample^ students in the traditional environments 
are likely to be presented information from textbooks, films, 
and lectures^ The information is often pre-packaged, the 
outcomes are specified in measurable terms , and there are 
agreed-upon cognitive structures for teaching the discipline- 
oriented subject matter. 



From the perspective of the traditional learning environment 
stufdents in the learning- in-work environments may be thought to 
be provided a series of random experiences, since the instruc- 
tional experiences cannot easily be categorized in. a prescribed 
curricoiilar fashien-i Information is acquired from stimuli in 
this environment tliat many times cannot be well specified, 
arranged, or measured. The worksite learning environment can 
be viewed as complex, temporary, and transitory. Student out- 
comes vary depending upon the unique relationships established 
among the student, learning coordinator, and worksite resource 
persons and within the parameters of the program goals. 

Retention of iRnow ledge 

Generally, studies indicate that the retention of 
knov/ledge rapidly decreases after it has been learned. For 
example;, Ebbinghaus studied his own retention of lists of 
nonsens^e .syllatbltes at differential time intervals after 
he had learned tliem. His results became knovm as the "forgetting 
curve.* " iSjaalysis of forgetting curves suggests that immediately 
after learning, individuals forget information rather rapidly, 
but that wi-th the passage of time, the rate of forgetting 
decrea'ses. An explanation of this phenomenon has been postulated 
suggesting t.hat two different memory systems determine an 
indiv^idual's capacity for short-term and long-term memory. 
According to this view, recently learned materials will be 
-quickly forgotten (less likely to be retained) unless they are 
practiced,, -whereas older, better-consolidated memories 
(experiences) may be maintained without the benefit of additional 
practice or rehearsal. 
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The research literatuare is replete with studies dealing 
with various aspects of retention. Much of the research details 
specific methods and paradigms designed to measure retention in 
short-term memory (STM) , which generally refers to time intervals 
of several seconds. Some researchers have introduced the 
notation of intermediate-term memory (TTM) to refer to time 
intervals of several minutes to perhaps an hour* Long-term 
memory (LTM) is, therefore, used to designate time intervals of 
more than one hour. The research usually occurs in a laboratory 
setting where there is control over the variables . thought to 
be related to retention such as the length of time between 
exposure and test, the presence or absence of cues, the degree 
of overtraining, the length of exposure to the stimulus, the 
nuiT±)er and nature of intervening rehearsals, and events occurring 
between exposure and test* 

For purposes of this study, our interest is in retention 
as it relates to long-term memory and as it occurs in naturalistic 
settings. Our intention is to examine further and to describe 
the relationships between the learning enyironments of students 
and their retention of subject-matter concepts. 

Summary of Related Research 

The learning-in-work approach vzould seem to offer the benefits 
of frequent, meaningful, and speedy feedback on task performance 
for students. Research in academic learning and retention 
(Ausubel, 1968; Boker, 1974; La Porte and Voss, 1975) would 
predict increased performance as a function of the above variables. 
Ausubel (1968) reported increased learning and retention of 
meaningful vs. rote memorized material. Anderson and Biddle 
(1975) and Boker (1974) found a strong relationship between 
increased application (practice) of material and subsequent 
retention. Similarly, La Porte and Voss (1975) demonstrated 
superior retention performance as a function of usage of the 
information and response-contingent performance feedback. 

As stated, the learning environment an individual chooses 
(learning-in-work vs. traditional classroom instruction) is 
important with respect to academic learning and retention; 
however, research has indicated person-related variables, (i.e., 
cognitive style) , interact with the environment, suggesting the 
difference in environment alone is not able to account for all 
differences in learning and retention. Davey (1976) concluded 
that the match of students' cognitive style, defined as stable 
preferences in individuals with respect to conceptual categori- 
zation and perceptual organization of the external environment, 
was a critical factor in maximizing performance. Hunt (1975) 
argued for future research directed at identifying the inter- 
active nature of the personal and environmental characteristics 
of behavior. 

5 
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Cronback and Snow (1977). advocated the consideration of the 
interac??on of instructional methods with individual charac- 
teristics, as they maintained that the traditional method of 
clasIroSm instruction was not optimal for all students. 
Itlu?fe??, Suedefeld and Driver (1965) found that more complex 
subjects were less influenced by the environment than less 
complex subjects.. In the same vein, Staszkiewicy (1977) 
indicated more cognitively complex students scpored higher than 
less complex students in situations characterized by less 
teacher direction. Consequently, a closer examination of the 
interaction between person variables (cognitive style) with 
iea?ning environment (EBCE vs. traditional classroom instruction) 
in the present investigation seeks to examine the contributions 
of each with respect tS academic learning and retention. 

- More detailed and specific information is found in Appendix 

A. 



Summary of Year-one Research 

Past research on learning- in-work programs has generally 

relied on the pre- test/post- test control 9^°^^? ^^ff^^^the results 
detect differences in student performance. Usually, the results 
T^H?^ii-e few statistical differences in the acquisition of 
subject^attef bei:een students in the ^ earning- in-work pro^^^^ 
and those in the control group. However, ^he authors believe 
thli- differences exist and that methodological and other 
problems ma? p?evS^i the detection of subtle changes (Crowe, 
lo?7 T979?! one reason for not detecting differences may be 
ihl^^ii ri^sf^arch design restricts the time required for 
differences ?oSmergl'^ That is, demonstrable student perfor- 
^ancefSrSchLvements may not occur until three to six months 
after program participation. 

To test this assumption, the design of this study uses 
repeated measures at the beginning (pre-test) and end (post-test) 
of the learning experiences and after the summer recess 
(follow-up test) . The time between the end of the program 
experiences and the end of the summer recess is considered as the 
?e?Jni!Sn phase of what students have learned. In other words, 
the patterns of student performances between post- ana 
fo?low-up ?es?ing may repeal the extent to which mathematical 
and reading concepts are retained after participation m one of 
the two learning environments. Thus, the design and 
in?erp?etation of results can be anchored to the broad concepts 
"of memory and retention phenomena. 
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The basic theoretical position adopted by the investigators- 
was that the learning environment to which students are exposed 
exerts a strong influence on subsequent levels of performance. 
To investigate some of the specifics of the interaction between 
students and their learning environments, two settings vzere 
selected which were seen to possess significantly different 
properties: (1) a "traditional" classroom environment, 
characterized by one teacher per class; subject matter organized 
by traditional discipline area (e.g., math, English); instruction 
confined to the classroom; and a schedule of five to six discrete 
instructional periods per day and (2) a "learning-in-work" 
environment, characterized by eighty percent of the student's 
time spent in community worksites with learning supervised by 
a "resource person" at each site; tv/enty percent of the student's 
time spent in a "learning center" where the student negotiated 
with a "learning coordinator" to plan his/her learning activities 
and evaluate his/her performance. A more thorough description 
of the Experience Based Career Education (EBCE) Program or what 
we have called a learning-in-v/ork environment can be found in 
Appendix B. 

The subjects for the first-year analyses were fifty-four 
students (twenty-seven for each environment) who were followed 
in their junior year (with tests at the start and end) and 
re-tested at the start of the following year. Accordingly, 
since the students in the learning-in-work program were roughly 
comparable to the students in the traditional environment, it 
was assumed that the design would afford a means to compare the 
performance of the two groups- and infer the "effects" resulting 
from participation in different learning environments. 

For the purposes of the preliminary investigation, the 
analyses used were primarily descriptive in nature. As a 
means to distinguish between environmental differences in learning 
and retention (as measured by performance on standardized 
academic tests of math and reading skills) , repeated-measures 
analysis of variance analyses were conducted over the three 
observation periods (the start and end of junior year, and the 
start of senior year) . A summary of these analyses is presented 
in figure 1, and a depiction of the performance means is given 
in figures 2 and 3. A more thorough description of the tests 
is found in chapter II. 

The results for the math concepts showed that, contrary to 
expectations, performance of the students in the learning-in- 
work environment demonstrated decre*asing slopes for the learning 
interval and increasing slopes for the retention interval. 
The retention of the reading concepts was essentially the same 
for both groups of students in that both groups showed 
increasing slopes for both the learning and retention intervals. 
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FIGURE 1' 
Year 1 Summary 
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FIGURE 2 



Student Performance on Reading Comprehension Scales for Three Testings 
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FIGURE 3 

Student Performance on Math Scales for Three-Testings 
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Patterns from student interviews and a prototype Learning 
Environment Questionnaire suggested that students in the 
learning-in-work environment perceived their learning environ- 
ment as more complex, when compared to students in the 
traditional learning environment. 

The results of the first year's analyses were interpreted 
in the context of a retroactive-interference design, which 
suggested that the rules for learning how. to learn in the 
work environment interfered with those acquired for learning 
in the traditional school environment. Thus, the decreasing 
slopes of the learning-in-work students observed for the 
learning interval could indicate the extent to which the rules 
learned in the traditional school environment interfered. The 
observed increasing slopes for the retention interval could 
indicate an integration of the rules for learning to learn in 
the work environment with rules for learning in a school 
environment. 



Problem Statement 

The grov/th of learning- in-work programs such as Experience 
Based Career Education, Career Intern Program, and the Executive 
High School Internships Program expands student learning 
opportunities to include workplace environments and this 
expansion provides young people the option of choosing one 
learning environment over another. Inherent in this choice is 

-the"assiimption~-that*---student:s"choosing--the-l.earni^ 

environment learn concepts associated with traditional subject 
matter as well as or better than students choosing a traditional 
learning environment. This assumption is based, in part, on the 
fact that students in the learning-in-work environments are 
exploring real problems in the workplace that relate to subject 
matter disciplines, are learning under the guidance of a mentor 
who solves work-related problems, are having experiences that 
pair subject matter concepts and specific work tasks, are 
learning concepts in a concrete "hands-on" manner, and presumably 
are forming cognitive structures for storing and retrieving 
concepts similar to those of the mentor at the workplace. 

Inspection of the results from the first wave of analyses 
led to two main conclusions. First, the performance patterns 
exhibited by the students on the reading scales of the CTBS 
were consistent with the expected results: the performance of 
both groups rose uniformly over time, with no significant 
differences between the group means at any of the testing 
occasions. It was assumed that these results illustrated a 
learning/retention function, demonstrating that students in both 
environments learned and retained skills in reading equally well. 
Math performance, however, did not conform to the pattern 
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observed; in the reading scales. Pronounced differences wi^re 
seen between the two groups of students at the second testing 
(at the end of one academic year in th.eir respective programs), 
which suggested that dissimilar processes were operating in the 
two environments with respect to learning/retaining/performing 
math skills. 

^. The present report constitutes an extension of the first 
' year's analyses based on the findings of differences in math 
performance between the two environments. The following general 
questions were raised. First, how did the environments differ, 
in terms of the students' perceptions of various characteristics 
which were common to both environments? Second, could support 
be obtained (through quasi-experimental analyses) for the 
hypothesis that different casual processes were operative as a 
function of differences in the learning environments? Third, 
were there student differences in the performance as a function 
of the types of worksite placements , and/or the types of 
summer activities? Finally, and of greater importance theo- 
retically, could the predicted interaction of personal character- 
istics (cognitive complexity) by environmental characteristics 
(environmental complexity) be obtained using math performance 
as the dependent variable? 

Thus, the general hypothesis to be examined is that 'there 
is relationship (interaction) between individual complexity 
and learning environment complexity (as a function of instruction 
received in traditional or learning-in-work environments) on 
the-^retention-'of—cognitive - skills . 



Research Framework 

Heuristic Model and Causal Relationships 

As a result of the preliminary examinations of the data and 
a desire to understand the cognitive processes and causal 
relationships underlying student learning in the learning-in-work 
environment, a heuristic model was used to guide the research 
questions for th,e present study. The model was used for two 
reasons: first, to permit questions to be asked in the context 
of causal relationships presumably operating for students in the 
learning-in-work environment; and secondly, to guide the analysis 
of the data. It should be noted that due to limitations of the 
data, the model is not being tested, but is rather being used as 
a road map to guide the current investigation. The heuristic 
model that represents the hypothesized causal relationships for 
students who are learning cognitive skills in a learning-in-work 
environment is as follows: 
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FIGURE 4 

Heuristic Model for Investigating Student Cognitive 
Processes in Learning-in-Work Environments 



Cognitive Learning Cognitive Learning 




Experience 
Key: 

> represents possible causal relationships 
represents measured variables 
mZZZ represents unmeasured variables 

Two general relationships are inherent in this model: 
(1) that the pre-test scores and all other unmeasured experiences 
and personal characteristics in association with the learning 
environment cause post-test performance to ocQur in such a way 
as to be related to a hierarchical achievement of cognitive 
skills as defined by Bloom's taxonomy (and as measured by the 
CTBS) ; and (2) that the post-test scores and all unmeasured 
experiences and personal characteristics in association with 
the summer environment combine to cause follow-up performance. 



Definition of Independent and Dependent Variables 

Before the specif ic research questions are described/ it 
may be useful to 'describe the measured independent and dependent 
variables. There are three classes of measures: environmental 
characteristics, personal characteristics ^ and performance 
effects. The environmental characteristics include (ij the 
worksite experiences both by number participating and as 
classified by the Worker Trait Group (WTG) from the Dictionary 
of Occupational Titles for the learning-in-work students;^ (2) 
the suiraner activities for both groups of students classified as 
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full- or part-time work or summer school or a combination of work 
and school, and (3) the students' perceptions of the learning environ- 
ment as measured by the Learning Environment Questionnaire (LEQ) . 
The personal characteristic of cognitive complexity was determined 
by the Paragraph Completion Test (PCT) , which resulted in a measure 
of cognitive style. The dependent variable was math performance 
as determined by the CTBS scores that are hierarchically developed 
along the lines of Bloom's taxonomy. 



Research Questions 

Based on the heuristic model for describing students' cognitive 
processes in learning environments and the measured variables 
obtained for students in this study, two research questions are 
investigated. Firsts student pre-test scores and exposure to 
either the learning-in-work environment or to the traditional 
learning environment caused math post-test performance (and 
follow-up performance) to occur in such a way as to be related to an 
hierarchical achievement of cognitive skills as defined by Bloom's 
taxonomy and as measured by the CTBS. That is, because the CTBS 
process scales of Recognition, Translation, Interpretation, and 
Analysis are hierarchically developed along the lines of Bloom's 
taxonomy, it is predicted that achievement of one process at 
time 1 will cause achievement of the next hierarchically-related 
process at time 2 as follows ( — represents causes): 



Pre-Test Post-Test Follow-Up Test 

Recognition 
Translation 
Interpretation 
Analysis 

Secondly, environmental characteristics v/ill exhibit an inter- 
active relationship with the student's personal characteristic 
of cognitive style in causing math performance. 

In order to augment the investigation of these two causal 
relationships, the factor structure of the LEQ was determined for 
both groups of students, as well as the extent to which the number 
of and type of site experiences classified by WTG were associated 
with math performance for the learning-in-work students. ^ Chap- 
ter II further delineates the research questions and their 
associated measured variables. 



Recognition 
Translation 
Interpretation 
Analysis 




Recognition 
Translation 
Interpretation 
Analysis 
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CHAPTER II 



DESIGN AND METHODOLOGY OF THE STUDY 



This chapter contains three sections that describe (1) the 
research background characteristics ^ (2) the description of the 
major measured variables (instruments)^ and (3) the analysis 
questions and strategies. 



Background 

Pre-Established Conditions 

The exploratory nature of this investigation determined the 
eventual design and methodology employed. Through the 
assistance of the National Institute of Education Project 
Officer f Ron Bucknam/ our staff located an AEL model of EBCE 
operating in Anoka ^ Minnesota. As part of the conditions of 
federal support for the initial operations of program^ 
a third-party evaluation was being conducted. Given the 
opportunity for National Center research staff to work 
cooperatively with the EBCE staff at the Anoka site and to 
"piggy-back" an on-going evaluation supported by federal 
resources f we accepted the pre-established .conditions of the 
third-party evaluation as part of the research design for 
this study. Additionally the staff of the Anoka-Hennepin School 
District No. 11 agreed to permit the testing of the evaluation 
students at the beginning of the following school year. Thus^ we 
were able to establish a repeated-measures design to investigate 
the learning and retention of concepts. 

The f ollpwing^ pre-estab^^ conditions of the third-party 

evaluation were included: 

• A pre- (September 1978) and post- (May 1979) testing 
of EBCE students and control students 

• Control students chosen at random from master school 
rosters to match the volunteer EBCE students in 
respect to membership in home high school^ grade 
point average/ and sex 

« Administration of the sub-tests of mathematical 
concepts / applications ^ and reading comprehension 
from the Comprehensive Tests of Basic Skills 
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- Finally, in order to ensure that follow-up testing 
conditions remained similar to those of the pre- and post- 
testing sessions and to minimize interruptions to the EBCE 
program/ the National Center contracted with the third-party 
evaluator to administer the follow-up tests (September^ 1979) 
at the same time that new students were being tested for 
the third-year evaluation. 



"Research Design 

Students could volunteer to participate in the EBCE 
program duiri-ng their junior year at high school. We 
designated this group as the learning-in-work program.^ 
The control students were juniors enrolled in the traditional 
program at one of the three high schools and randomly 
selected from a master school roster to match the 
learning-in-work students as to the characteristics of 
membership in the home high schools , grade point average, and 
sex. This second group we designated as the traditional 
learning program. 

The independent variables were the two learning environment 
(programs) and the learning and retention intervals (time). 
The dependent variables were student scores on the sub-tests 
of mathematical concepts, application/ and reading comprehension 
of the Comprehensive Tests of Basic Skills. Both content 
scores and process dimension scores were yielded by these 
measures. 

The design for this investigation is probably best 
described as quasi-experimental , since -students were not 
randomly assigned to participate in the EBCE program. Pic- 
torially, the design for this study is shown in Figure 5. 



FIGURE 5 
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The treatment Xi is the learning-in-work environment while 
X2 is the traditional learning environment. The treatment X3 
is the summer recess, and it is assumed that the experiences of 
the students would be similar. The design is a repeated-measures 
one where the nature of the observations (0) is defined as 
follows (figure 6) : 



FIGURE 6 
Descriptions of the Observations 
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General izability 



. The lack of random selection of EBCE students affects 
the generalizability of the study's results. Because of the 
random selection of the control students and the extent to 
which the variables used for matching do not decrease 
representation r we would expect a reasonable ability to 
generalize to junior students from the same general background 
(e.g region of country r socio-economic status) . However, 
the fact that random assignment was not used to place students 
the learning-^in-work program places limitations on the extent 
to which generalization of the observed results of this EBCE 
program can be made to other samples/populations of students 
participating in an EBCE program. Because the students 
were "self-selected" for membership in the learning- 
in-wprk program, we do not know the extent to which they 
are atypical of the "average" high school student. Overall, 
it would not be prudent to state that our obtained 
observations can be expected for a group randomly assigned 
to the program. Essentially, for the learning- in-work group, 
we can only describe the way things are for our sample but can 
not predict for a larger population with any substantial 
degree of conf idence . To obtain that degree of generalizabili' 
a "true experiment" would be necessary in which students are 
randomly assigned to both the traditional and EBCE programs. 



Data-Collection Procedures 

Data were collected in September, 1979; late May, 1979; 
and September, 1979. The pre- and post-testing was 
performed by Evaluation Consultants under a contract from the 
school district to conduct a third-party evaluation of EBCE. 
The follow-up testing was also conducted by Evaluation 
Consultants under a contract from the National Center. 

Students from both programs were assembled at their 
home schools for testing, which proceeded according to CTBS 
guidelines. Therefore, testing conditions and times were 
identical for both the traditional and learning-in-work 
students (for each home high school). The LEQ was 
administered at the post- testing session. A make-up session 
following the above procedure was held for each observation 
to ensure the maximum number of responses. Testing time 
required about one hour. 
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Sample Size 



Table 1 indicates the number of students available for 
testing at the- time of the three testing observations, 
categorized according to their sex and home high school. 

As can be seen, there was a 27.5 percent overall subject 
mortality, since only students v/ho had complete instruments for 
all three testings were included in the analysis. 



TABLE 1 
Size of Sample 



Pre Post Follow- Up* 



1 

\ 
1 

iTotal Nuxnber of 
Students Tested 


Learning- 
in-Work 
Environment 

38 


Traditional 

Learning 
Environment 

42 


Learning- 
in-Work 
Environment 

38 


Traditional 

Learning 
Environment 

42 


Learning' 
in-Work 
Environment 

30 


Traditional 

Learning 
Environment 

23 


iNo. o£ Males 

i 


16 


23 


16 


23 


11 


13 


|no. o£ Fanales 

i — 


22 


19 


22 


19 


19 


15 


!no. from High 
:School »1 


10 


14 


10 


14 


8 


9 


No. from High 
School #2 


14 


13 


14 


13 


10 


11 


No. from High 
School #3 


14 


15 


14 


15 


12 


8 



• Subjects for the analyses were drawn from these totals, but the usable sample 
size is smaller (T»«27, C-27) than the total number surveyed/ due to four 
subjects having inccmplete data on pre-tcst and/or post-test. 
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Instrumentation 



The instruments used for the investigation are described 
here. In addition to using the CTBS to measure student 
achievement. of subject-matter content, the project staff, with 
the assistance of Dr. Harold Schroder, of the University of 
South Florida , developed an experimental instrument designed 
to describe student perceptions of the complexity of the learning, 
environment (Learning Environment Questionnaire, LEQ) . Finally, 1 
staff used the Paragraph Completion Test (PCT) to obtain a 
measure of student cognitive complexity. The CTBS was 
administered at pre-, post-, and follow-up testings; the 
LEQ and PCT^- at the post-test only. 



Comprehensive Tests of Basic Skills 

The test used to measure student retention of concepts was 
the Comprehensive Tests of Basic Skills (CTBS), Expanded Edition, 
level 4, form S, Test 2 — Reading Comprehension; and Test 7 — 
Mathematics Concepts and Application. According to the CTBS 
literature, the instriament. measures skills and concepts. The 
Test Coordinator's Handbook (1967) describes the test as a 
measuring instrument thus: 

Measurement: of the basic skills and abilities cannot 
be divorced entirely from the measurement cf knowledge 
acquired through schooling, but it is not the intent of 
CTBS to measure this knowledge directly. The 
emphasis . . . is on measurement of the grasp of broad 
concepts and abstractions as developed by all curricula 
and on the facility in the skills that are required for 
effective use of language and number . . . * . 

To provide the readers with an overview of the instrument, the 
following excerpts from The Coordinator's Handbook are 
presented: 

Rationale . CTBS, Expanded Edition, was designed to 
measure the extent to which individual students have 
developed the capabilities and learned the skills that 
are prerequisite to studying and learning in subject- 
matter courses .... 

The basic skills are developed through exposure to a 
variety of curricula and instructional procedures • • • 
these tests of basic skills are not greatly affected 
by the particular content material used to teach 
students. 
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The emphasis • . . is a measurement of the grasp 
of broad concepts and abstractions as developed 
by all curricula . • . . 

Test items . In the development of the tests, 
attention was given to long-term trends in 
curricula . . . items were excluded that could 
not be assumed to be based on common experience. 

(through research), CTBS/M-cGraw Itill has r 
attempted to identify and eliminate racial and - 
ethnic bias .... 

Classification schem .e. The (classification) 
scheme is so arranged that there is an. 
inherent hierarchy of skills, or levels of 
thinking, consistent with what is currently 
known about child development, cognitive 
structures, and learning processes. 

The process classification of items for the CBTS 
follows essentially the approach presented in 
Taxonomy of Educational Objectives , edited 
by Bloom. This approach has been adapted to 
the CTBS design. Two major components provided 
the basis for devising the classification 
scheme for CTBS: the "process" dimension and the 
"content" dimension. 

Process dimension. The emphasis in the process 
dimension is on the' measurement of comprehension 
and application of concepts and principles rather 
than on the measurement of knowledge per se . • . - 
the tests measure knovledge of ways and means 
of dealing with these concepts and principles as 
reflected in applying rules . . . and processes. 

The classification scheme is hierarchical in nature 
as the level increases, more complex processes 
are used with greater frequency than simpler 
processes. There is also an increase in complexity 
and abstractness of the test items over the 
levels . ; • . 

Content dimension . The categories in the content 
dimensions differ • • - . among the various levels 
of CTBS. The type of content was selected at 
each test level for its appropriateness to the 
students of the grades for which it was intended • 
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Content and process relationships . Figure 7 provides a 
description of the Content dimensions, for the mathematics 
concepts and -application tests. Figure 8 is the description 
of the Process dimensions that are applicable to the CTBS 
tests and that are related to Bloom's Taxonomy of Educational 
Objectives. Figure 9 describes the relationship of the 
mathematics concepts and application content categories to 
the process dimensions. The major purpose of these figures is 
to facilitate an understanding of the overall rationale / 
logic, and structure of the CTBS. Furthermore, they are 
intended to demonstrate that what is being measured, in addition 
to the mathematical content, is cognitive processes or 
operations underlying learning and memory. 

Reliability and validity of CTBS . CTBS reliability and 
validity coefficients are obtained from the CTBS Technical 
Bulletin No. 1. Kuder-Richardson formula 20 (KR 20) 
equations were used to determine the reliability of the CTBS, 
and constitute an estimate of the tests' internal consistency. 
Reliability measures reported by CTBS are as follows : 
Reading Comprehension (r = .91); Math concepts (r = .85); 
and Math Applications (r = .88). These values are quite 
acceptable, and they presumably reflect the precision with which 
the CTBS scales were constructed. Content validity, or the 
extent to which the test samples a representative group of 
items from the total content domain, was assessed by a 
variety of procedures , including consistency of items with 
the total scores and the ability of the items to discriminate 
between achievement groups of students. Reading items were 
selected to measure critical skills of theme identification, 
character analysis, and inference. Math items were selected 
to measure concept recognition, selection of proper 
problem-solving strategies, and implementation of these 
strategies. 
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FIGURE 7 



Description of the CTBS Content Dimensions 
for the Mathematics Concepts and Application Tests 



C oncept Dimension Categories 
Number/System/Properties 



Geometric Relationships 



Sets 



Graphs 



Mathematical Sentences 



Problem Solving 



Reasoning 



Description 

Convert from one mathematical 
form to an equivalent form 

Understand relationships of 
comparisons, number properties, 
and' place value 

Recognize curves, plane 
shapes 

Understand formulas 

Compute measurement of figures, 

given the formulas 

Recognize, estimate, compare, 
compute measures of time, 
length, money, weight, area 

Understand set relationships 
Determine sets described in 
problem 

Interpret information from 
circle or line graphs 

Interpret given point on graph 

Solve for unknown values in 
equations 

Comprehend a problem 

Select method to solve problem 

Determine information needed to 
solve problem 

Determine logical conclusions 



Note: Adapted from CTBS Test Coordinator's Handbook, p. 18. 
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FIGURE 8 



Description of the CTBS Process Dimensions 



Process Dimension Categories 
Recognition 

Translation 



Interpretation 



Applicat ion 



Analysis 



Description 

- Recognize, or recall and apply 
factsv theories, concepts, 
structures previously learned 

- Transform concepts and symbols 
into equivalent forms 

>- Interpret graphically presented 
data 

-Understand facts and concepts in, 
and make inferences from, 
written material 
^ -"^Associate data and ideas 

- Comprehend relationships 

- Summarize major themes or 
concepts 

- Apply prior knowledge of facts to 
solve problems 

- Use previously learning research 
skills to determine meaning of 
new material 

- Apply formal logic to synthesis 
of organizational patterns or 
components 

- Determine hierarchical arrangement 
of facts 



Note: 'Adapted from CTBS Test Coord inator's Handbook, p. 
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FIGURE 9 

Relationship of Mathematics Concepts 
Content Categories to Process Dimensions 



Process Dimensions 



Concepts 

Content 

Categories 



Recognition Translation 


Interpretation 


Analysis 


Niomber 


- Number Sys- 


- Math 


Systems 


tems 


Sentences 




- Sets 


- Geometric 




- Measure- 


Relation- 




ment 


ships 




- Geometric 


- Problem 




Relation- 


• Solving/ 




ships 


Reasoning 




- Problem 






Solving/ 






Reasoning 





' Relationship of Mathematics Application 

Content Categories to Process Dimensions 

Process Dimensions 

, Recognition Translation Interpretation Analysis 

Applicatior 

Conteiit "Measurement -Measurement -Problem 

Categories -Math Solving/ 

Sentences Reasoning 
-Geometric. 

' Relationships 

. -Problem 

Solving/ 
Reasoning 



Note: Adapted from CTBS Test Coordinator's Handbook ^ p. 19. 



Learning Environment Questionnaire (LEO) 



Because the investigation is concerned with learning and re- 
tention in two learning environments, a measure to assess student 
perceptions of the complexity of the environment was developed. 

Two constructs were used to develop the LEQ: the student 
perception of environmental complexity and the degree of per- 
ceived internal or external control over learning, tasks. Environ- 
mental complexity was considered in terms of different types of 
environments, one in which concepts are generated and another in 
which concepts are presented. Indicators of concept generators 
may include opportunities to observe, ask questions, experiment, 
develop ideas, and evolve alternative problem solutions. Indicat- 
ors such as these can be characteristic of an open environment 
where individual*? are supported and encouraged by the environment 
to achieve goals. On the other hand, indicators of environments 
which present concepts may include those in which learners are 
told what to do, are taught the correct or acceptable way. to 
handle a situation, and are given a little encouragement to be 
creative in solving problems. Indicators such as these may be 
characteristic of a closed environment * where individuals perceive 
the environment as nonsupportive of achievement of individual goal 
A concept-generating environment may be considered more complex 
than a concept-giving environment, in part because decisions must 
be made v/ithout the benefit of readily available rules and prede- 
termined, consequences. Thus-, individuals in these environments 
may have to consider more dimensions and 'make more judgements 
with finer discriminations to decide how to solve a problem. 

Closely related to the concept of complexity is the degree 
to which individuals feel internally motivated for work. That is 
if individuals believe that they are doing a good job in meaning- 
ful work and that they alone are responsible for completing the 
task, they may take their cues from an internal "sense of needs" 
for task performance (internal control) . Conversely, individuals 
who perceive that they have limited opportunity to do meaningful 
work or solve problems and that they have limited control over 
decisions may take their cues from the environment for task 
performance (external control) . 

For purposes of* this investigation, we would expect the 
learning-in-work environinent to be more concept-generating than 
concept-giving and would expect the students to perceive more 
internal control than external control. 

Figures 10 and 11 present the LEQ for the EBCE students 
and the students in the traditional learning environment respect- 
ively. The difference in the two LEQS accommodates the specific 
actors who guide the student learning (learning coordinator as 
compared with classroom teacher) in the two learning environments 



Learning Environment Questionnaire 



This questionnaire asks about your learning situation in the ExperienceTBased Career Education program. The 
purpose is to find out your perceptions of situations where.you learn and your reaction to them. There are 
no right or wrong answers. Place a check 2] the cell □ ithat best represents your opinion. 

Strongly Moderately Moderately Strongly 

Disagree Disagree Undecided Agree Agree 

1. In the EBCE program I felt encouraged to find □ □ □ □ □ 
things out for myself 

2. I was able to tell by myself if I was doing a good job □ □ □ □ □ 

3. The learning coordinator taught me what I needed □ □ □ □ □ 
to know 

4. The resource person taught me what I needed CD □ □ □ □ 
to know 

5. In the EBCE program I was able to ask many □ □ □ □ □ 
questions about the work 

6. The results of what I did had meaning. I felt the □ □ □ □ □ 
results were important 



7. The learning coordinators described the way they 
wanted me to do my work 

8. The resource persons described the way they 
wanted me to do my work 

9. In the EBCE program I had opportunities to try 
things out for myself 

10. The work that I did offered me many different □ □ □ □ □ 

things to do 



a □ □ □ □ 

□ □ □ □ □ 

□ □ □ □ □ 



i i . The learning coordinator gave me the right 
way to do the work 



□ □ □ □ □ 



12. The resource person gave me the right way □ □ □ □ □ 
to do the work 

13. In the EBCE program I was encouraged to □ □ □ □ □ 
come up with my own ideas 

14. The resource person provided me opportunities □ □ □ □ □ 
to do meaningful work or solve problems 

[□□□□□ 

□ □ □ □ □ 

□ □ □ □ □ 

□ □ □ □ □ 

□ □ □ □ □ 



15. The learning coordinator provided me opportunities 
to do meaningful work or solve problems 

16. The learning coordinators showed me what they 
required me to do 

17. The resource persons showed me what they 
required me to do 

18. The resource person encouraged me to decide 
for myself how I was going to do my work 

19. The learning coordinator encouraged me to de- 
cide for myself how I was going to do my work 
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FIGURE 11 
Learning Environment Questionnaire 

This questionnaire asks about your learning situation in your school program. The purpose is to find out what 
you think of situations where you learn and your reaction to them. There are no right or wrong answers. Place 
a check El Jn the cell □ that best represents your opinion. 



1. In my school program I felt encouraged to find 
things out for myself 

2. 1 was able to tell by myself if I was doing a good 
job 

3. The teachers taught me what I needed to know 

4. In my school program I was able to ask many 
questions about the work 

5. The results of what I did had meaning. I felt the 
results were important 

6. The teachers described the way they wanted me 
to do my work 

7. In my school program t had opportunities to try 
things out for myself 

8. The work that I did offered me many different 
things to do 

9. The teachers gave me the right way to do the 
work 

10. In my school program I was encouraged to come 
up with my own ideas 

11. The teachers provided me opportunities to do 
meaningful work or solve problems 

12. The teachers showed me what they required me 
to do 

13. The teachers encouraged me to decide for 
myself how I was going to do my work 



Strongly Moderately Moderately Strongly 

Disagree Disagree Undecided Agree Agree 

□ □ □ □ □ 

□ a n □ □ 

a □ □ □ □ 

□ □ □ □ □ 

□ □ □ □ □ 

□ □nan 

□ anno 

□ □ □ □ □ 

□ □ □ □ □ 

□ D □ □ □ 

□ □ □ □ □ 

□ a □ □ □ 
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Paragraph on n^i=>c;-H (vr^ ) 

The Paragraph Completion Test (PCT) derived from the 
work of Schroder, Diver, and Streufert (1967) represents an 
approach to the content-free measurement of cognitive style. 
This approach involves a projective technique of stem 
completion. The PCT presents the subject with five stems. 
Three sentences are to be written for each stem. The integrative 
or cognitive complexity score used for this study was the 
mean of the four most abstract responses after each stem had 
been scored on a seven-point scale. The stems used for this 
study were as follows: 



1. Your friend: 
Your answer : 

2. Your friend: 
Your answer : 

3. Your friend: 

Your answer: 

4. Your friend: 
Your answer: 



Rules! How do you feel about rules? 
Rules .... (complete the sentence and 
write 2 more . ) 

Sometimes I feel doubt. When you are 
in doubt, how do you feel? 
When I am in doubt .... 

Sometimes my teacher disagrees with me. 
How do you feel about disagre:ement with 
your teacher? 

When someone disagrees with me ...» 

Sometimes I feel confused.. How do you 
feel when you are confused? 
Confusion .... 



Your friend: 
Your answer : 



Sometimes I am criticized. How- do you 
feel when your teacher criticizes you? 
When others criticize me it usually 
means .... 



The PCT was scored by Dr. Harold Schroder and associates. 

An example of a concrete response to the stem "When I 
am criticized . . . is "I feel happy." "Anyway, I don't 
listen too carefully to criticism. I guess I'm occupied with my 
own little world, so outside criticism isn't very important 
to me" (Schroder and others, 1967, p. 192). An example of 
an abstract response to this stem is "I listen carefully. 
Criticism tells me much about the criticizer how he thinks, 
what he believes in, what he expects of others. It also 
tells me how others see me. After that,. I usually find myself 
changing my way of acting or thinking in order to take this 
into account" (Schroder and others, 1967, p. 193). 
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Analysis Considerations 

Analytic Strategy 

The present analyses were performed in order to address 
questions" raised in the preliminary examination of the data. 
These questions can be profitably viev/ed in the context of an 
overall model of the causal processes presumed to be operative 
for this study. A graphic presentation and description of this 
model is depicted in chapter figure 4. 

The model uses notation borrowed from the path-analytic 
literature (e.g./ Kenny^ 1979; Kerlinger and Pedhazur, 1973). 
VThile the use of this general model v;ould seem to suggest an 
analytic strategy including estimation of path coefficients 
for such a model (i.e./ a "path analysis"), this approach 
was rejected for the follov/ing reasons . First, and foremost, the 
present study did not allow the measurement of the number and kind 
of variables nor the measurement at specif ic observation periods 
desirable for a proper causal-modeling approach, due to con- 
straints imposed by the particular characteristics of the sample 
selected . for the study. For example, measures on variety of 
background variables (e.g., T.Q., family structure, scholastic 
ability, student personality) would have been highly desirable 
prior to the initiation of the study and the formation of the 
groups. Additionally, both the small size of the sample, as 
well as the non-random assignment of subjects to the learning 
environments, would make the obtained parameter estimates both 
highly unstable and of questionable generality to the population 
of students eligible for participation in such programs. 

Another common technique, the analysis of covariance, was 
also considered and rejected for these data. While the temptation 
was present to use "adjusted" means 'to assess the effect of the 
different educational environments (e.g., partialling such vari- 
ables as sex, GPA, and pre-test score from the post-test and/or 
follow-up) on math and reading achievement, this strategy was 
deemed inappropriate. As is noted by Cohen and Cohen (197 5, 
pp. 395-397), the potential for dual-direction causal relationships 
using existing, self-selected groups) and the concomitant 
problem of dividing the shared variance' between these research 
factors would be expected to produce coefficients (and adjusted 
means) which would be at best mathematically unjustified, and at 
worst seriously misleading. The above-mentioned types of questions 
can only be addressed, in a valid fashion, through use of some- 
thing approximating a "true" experimental design which includes 
experimenter control over assignment to treatment conditions, 
thereby guaranteeing no two-v/ay causality between the research 
factors and the covariates. 
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TABLE 2 

Existing-Gfoup Differences on Background 
Variables, on CTBS ^Scalen 



Pre-Test 
D.V.: CTBS 



Applications 



Intercept 
Sex 

Number of Previous 

Math Classes 
GPA 



D.V,; CTBS - Concepts 

Intercert 
Sex 

ITunher of Previous 

rath Classes 

GPA 



Parameter 



1.12 
-0.14 



4.76 
3.43 



-4.40 
0.95 

7.06 
2. 



0.39 
-0.14 

3.02 
4.84 



-1.24 
0.78 

3.52 
2.70 



,70 
,89 

.004 
.0001 

.48 



22 
44 

0009 
009 



= .37 



Support for this position can be seen in Table 2. These 
differences would suggest that the two groups, as a function of 
the self-selection process, were not comparable and that there 
was good reason to suspect that whatever factor (s) caused the 
self selection could be related causally to the covariates (e.g., 
GPA, number of previous math classes). Thus, while the groups 
were not technically formed until the start of observation, the 
suspicion which precluded such techniques as analysis of covari- 
ance was that these groups "existed" prior to membership in the 
learning-in-work program; accordingly, it would be impossible to 
state that the covariates caused the observed differences in math 
performance as a function of the program. 



Analysis Questions 

Accepting the limitations imposed by the measured variables and 
the sampling strategy but desiring to ask provocative questions 
arising from the preliminary analysis of the data, we adopted 
the following analytic approach phrased in terms of five 
questions : 



1 . ^That is the factor structure of the learning 



environment rruestionnaire (LEO) ? Questions concern 



• the underlying dimensions measured by the LEQ including 
similarities and differences between student perceptions 
of the role of the learning coordinators, the on-site 
resource persons, and teachers. 

TI . How does Bloom's taxonomy as measured by the CTBS 

function for students in the two learning environments ? 
Questions concern the applicability of the presumed 
Bloom predictions of a hierarchical structuring of 
scales (as defined and measured by the CTBS) in both 
learning environments, as well as the factor al structure 
of the CTBS with respect to measuring the Bloom "factors." 

IXT. What is the effect of type-of -placement (as defined by 
the: IVTG of the DOT) on the learning-in-work student's 
performance ? Questions in this section are concerned 
with the association between exposure to different worker 
trait group jobs at the students' placements (treatment 
students only) and subsequent performances on the CTBS. 

IV. What is the effect of type of summer activity on follow- 
up performance ? -In this section , questions concerning- 
the relationship between students' summer activities and 
follow-up performance aire addressed. For example, was 
there a difference between students enrolled in formal 
coursework over the summer recess and those who were not, 
and was there a difference between students who worked 
over the summer break and those-- who didn't work. 

V. Is there. a moderating relationship involving the LEQ and 
the student's cognitive style ? This question is concerned 
with the prediction of the authors that the relationship 
of the learning environment (perceptions) to performance 
will vary as a function of the student's cognitive style. 
More specifically, an interaction between cognitive 
style and the students' perceptions of the learning 
environment is predicted. 

Chapter TIT describes the analysis of the findings for 
these five questions. 
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CHAPTER II T 

..FINDINGS 



Overview of Chapter 

This chapter presents the findings for the research 
questions described in- chapter I. Figure 12 is presented as 
an organizing aid to help structure the discussion in this chapter 
The chart depicts the independent and dependent variables and 
the associated analysis/research question (s). Pxeceding the 
discussion of these questions, the ^'EQ is examiined for its 
underlying structure. The chapter concludes with an overall 
summary and discussion of the research findings* 

FIGURE 12 



Relationship of Independent and D'ependen.t Variables 
to the Analysis and Research Questions 



Independent Variable Dependent Variable 



Learning- in- work 
and traditional 
learning students, 
and time intervals 
(T^VsT^/ T^VsT^r 

T^Vs'r^) 



CTBS math scores as 
defined by the pro- 
cess variables based 
on Bloom's taxonomy 
and measured at post- 
and follow-up testing, 
The hierarchical 
arrangement of the 
CTBS process (Blooms' 
taxonomy) scales are 
Recognition , Trans- 
lation , Interpreta- 
tion, and Analysis. 



Analysis /Research Question 

Did being in one of the two 
learning environments cause 
students; to learn math skills 
as predicted by* Blaom'."s^ 
taxonomy?' According to- CTBS, 
the math items were developed 
along the lines of Bloom;' s 
taxonomy, therefore, certain 
predictions, can. be evolved - 

(a) Mastery of ..Recognition 
items at time 1 causes 
an' increase in Trans- 
latxon items- at time 2. 



(b) Mastery- of Translation 
items at time 1 causes 
an increase in Inter- 
pretation items at time 2 



(c) Mastery of " Interpretation 
items at time^ 1 causes an 
increase- in Analysis 
items at time 2. 



FIGURE 12 (cont.) 



Number of site 
experiences, type 
of work experi- 
ence by WTG for 
learning-in-work 
students 



CTBS math scores of 
content and applica- 
tion at post- and 
follow-up testings. 



Did the type of worksite 
exposure as defined by WTG 
cause math achievement for 
students in the learning-in- 
work environment? 



3. Learning-in-work 
and traditional 
learning stu- 
dents, summer 
activities 



CTBS math scores of 
content and appli- 
cation at the 
follow-up testing. 



Did summer experiences 
contribute to math achieve- 
ment and the end-of- summer 
(follow-up) testing for both 
learning groups? 



LEQ and PCT 



CTBS math scores of 
content and applica- 
tion at post:- and 
follow-up testing. 



Is there a main effect or 
interaction for perception 
of learning environment 
(LEQ) with cognitive style 
(PCT) for math achievement? 
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Factor Structure of LEO 



Results 

In order to examine the underlying structure of the percep- 
tions of the learning environment/ as reflected in the items of 
the LEO (figures 10 and 11) / separate-factor analyses were performed 
for the set of LEQ items, including the item stems "learning coor- 
dinator" and "resource person" for the students in the learning-in- 
work environment. These stems were paired in each analysis with 
the corresponding "teacher" stems for the students in the, "tradi- 
tional" learning environment, whenever the stems were not judged 
identical in content (i.e., for LEQ items 3, 6, 9, 11, 12, 13). 

Principal axis factor extraction, using the parallel analysis 
procedure/ was performed on the correlation matrices generated from 
the data described above in order to generate eigenvalues for 
comparison, which indicated retaining three factors for each 
analysis. Squared multiple correlations (SMC) were used as 
community estimates for all analysis models* The common-factor 
model was chosen in view of the belief that a model which factors 
only common variance would provide the most reasonable and. real- 
is tic view of our data. To obtain the final-factor solutions, 
iterated principal axis factor techniques were used with SMCs as 
initial estimates of the communalities , retaining three factors. 
Because orthogonal (uncorrelated) factors would seem to be an 
unrealistic constraint to impose on these solutions, Proxmax (K-3) 
oblique transformations of the factor-loading matrices were 
performed to produce a final solution for each set of item stems. 
The results of these analyses are presented in Table 3. 

Comparison of the factor solutions was also accomplished by 
the calculation of coefficients of congruence, which provide a 
measure of the similarity of factors, as well as by calculation 
of root mean squares/ which correspond to "distances" between 
corresponding elements of each vector. The results of these 
comparisons are presented in Table 4. 

It is apparent from these results that the first factors in 
both solutions, (i.e., based on "coordinator" and "resource person" 
items) are very similar/ although the second and third factors 
exhibit some degree of difference. Hov/eyer, for interpretative 
purposes, both solutions would yield virtually identical classifi- 
cation of items. 



Interpretation 

It was originally predicted (Crowe and Harvey, 1979) that two 
factors were addressed by the LEQ: complexity of the environment 
(e.g,, the degree to which the environment v/as perceived 'as generating 
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Pr omax CK=3) 
LEQ 



TABLE 3 
LEQ Factor Solution 
Primary-Factor Pattern Iioadings 

Resource Person I,earning Coordinator 





Final h"^ 




II 


III 




T 


TI 


III 


(4) 


Able to ask 
questions 


.7t) 


.67* 


.00 


..24 


.71 


.72* 


-.07 


1 .22 


(5) 


Perceived iippor-- 














• 






tance/meaning in 
results 


• 55 


.36* 


.36* 


.14 


.50 


.46* 


.11 


.25 


(8) Many things t.o do 




, 68* - 


.12 


,33* 


.68 


.60* 


.16 


.38* 


(10) 


Enco^iraged to Acome 
up wl*th ideas of 

' own ■ . i. ; 




.67* 




,17 


.13 


.81* 


-.02 


.07 


(11) Had opportuniti^es 
i to ^do ^meanlTigful 


.68^ 


.72* 


.25 - 


.-..12 


.64 . 


.78* 


.18 


-.14 


(13 ) Able *o decide how 
. to 'do worfc 




. 75* 


.03 


-.18 


.42 


.52* 


.26 


-.07 


(3) 


Taught me xwflialt T 
needed to know 


• bo 


.28 


.48* 


.14 


.59 


.35* 


.46* 


.11 


(6) 


Way fco do work 
described to me 


.44 


- . 08 


.60* 


.17 


.44 


-.14 


.71* 


. 04 


(9) 


"Gave me right tway 
to ao the work" 




.21 


.67* 


.02 


.64 


.26 


.66* 


-.04 


(12) 


"Showed jne what was 
required to ao'" 


' .53 


. 00 


.70* 


.04 


.47 


.08 


.54* 


.19 


(1) 


Encoaartaged to find 
things (Out for 
myself 


.61 


.07 


.12 


.75* 


.60 


.00 


.05 


.75* 


(2) 


Able to tell if 
doing -a ^ood iob 


.S4 


.06 


.15 


.68* 


.60 


-.05 


.16 


,71* 


(7) 


Able to try thxng:s 
out by myself 


.64 


.30* 


.02 


.56* 


.64 


.37* 


-.04 


.53* 



Factor vCorrelations 



Resource 
Person 





T 


II 


III 




1.00 


.62 


.63 


II 


.62 


1.00 


.58 


III . 


.63 


.58 


1.00 



Learning 
Coordinator 



N = 93 

Denotes loadings 
ber^ship factors. 





I 


II 


III 


I 


1.00 


.49 


. 62 


II 


.49 


1.00 


.37 


III 


.62 


.37 


1.00 


and 


item 


mem- 
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Coefficients of Congruence 



TABLE 4 
Factor Comparisons for LEQ 
1 



Resource I 
Person II 
III 

Root Mean Squares' 



Resource I 
Person II 
III 



Learning Coordinator 
I II 



.997 
.047 
.027 



.051 
-.458 
-8 .160 



Learning Coordinator 
I II 



.-063 
.711 
.709 



-■750.,,.;. 

.428 
.457 



III 



022 
821 
496 



III 



,748 
132 
361 



'High values indicate greater similarity 



'Small values indicate greater similarity 
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or giving (concepts and rules); and control (e.g., the degree to 
which the students percoived the environment- as offering oppor- 
tunities to initiate tasks, solve problems, and negotiate work with 
adults) . It was hypothesized that the items comprising the 
complexity factor were 1, 4, 7, 10, 3, 6, 9, 12 while the remain- 
ing items comprised the control factor. As can be seen in Table 3, 
both similarities and discrepancies were observed with respect to 
the original predictions. Three factors appear to underlie the 
structures of the LEQ. Inspection of the items classified with 
"significant" loadings showed a general consistency with the 
predictions. Thus, while the original predictions were not totally 
supported, the factor solutions of LEQ items were both impressive 
in terms of magnitude of the loadings as. well as the "cleanness 
of the rotated solutions. 

Summated scales based on these above classifications were 
formed to provide measures based on the factor analysis. Results 
of correlational comparisons of these scales between the two groups 
are reported in Table 5. 

As can be seen in Table 5, the students in the learning-in- 
work environment (coded as "1") were significantly and consistently 
higher on these scales than the controls in the traditional 
environment (coded as "0"). In these analyses, the mean difference 
between the groups is given by the parameter for the program, 
variable, expressed in raw LEQ scale units; the t value for this 
quantity is the test of the significance of this difference between 
means. These results are consistent with the item-by- item com- 
parisons reported in the first year's report. 

Given v,;he factor solution of the LEQ, the following items 
were classified under factor I: 

(4) ... I v/as able to ask many questions about the work. 

(8) The work that I did offered me many different things to do. 

(10) ... I was encouraged to come up with my own ideas. 

(11) (the adult) provided me opportunities to do meaningful 
work or solve problems. 

(13) (the adult) encouraged me to decide for myself hov7 I was 
going to do my work. 

and possibly: 

(3) (the adult) taught me what I needed to know. 

(5) The results of vrhat I did had meaning. I felt the results 
were important. 

• (7) ... . I had opportunities to try things out for myselt . 
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TABLE 5 



Prediction of LEQ Scores from Program Membership 
Treatment 

1 



LEQ' I 
LEQ II 

LEQ III 



mean 
4. 30 
4.02 
4.42 



std 
.58 
.72 
.73 



range 

3-5 
1.75-5 
1-5 



Controls 
LEQ I 
LEQ II 
LEQ III 



3.66 
3.68 
3.76 



.73 
.55 
.58 



1.8-5 

2-5 
2.3-5 



LEQ I 



LEQ II 



LEQ III 



Intercept 
4 

Program 
= .194 



Intercept 
Program 

= .067 

Intercept 
Program 
R^ = .204 



Parameter 

3. 66 
0.64 



3.68 
0.34 



3.76 
0.66 



4.38 



2.40 



4.53 



,0001 



02 



.0001 



ERIC 



All scores were stunmed, then divided by the number 
of items to .give mean values for the. scale scores. 

'Possible range was from lowest = 1 to highest = 5. 

= 82 

^Treatment students coded as 1; control students coded as 0 
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The following items were classified under factor II: 



(3) (the adult) taught me what I needed to know. 

(6) (the adult) described how T was to do my work. 

(9) (the adult) gave me the right way to do the work.- 

(12) (the adult) showed me what was required of me. 

and possibly: 

(5) The results of what I did had meaning. I felt the results 
were important. 

The following items were classified under factor III: 

(1) . . .1 felt encouraged to find things out for myself. 

(2) I was able to tell by myself whether I was doing a good Dob. 

(7) ... I had opportunities to try things out for myself. 

and possibly: 

(8) The work that I did offered me many different things to do. 

Although the LEQ requires additional research for validation, 
the analysis suggests that it does discriminate between student 
perceptions of learning-in-work and traditional learning environ- 

., ments With the additional information from the factor analysis 

of student responses, the three factors may be interpreted as 
students' perceiving (1) the environment as providing an oppor - 
tunity for generating- concepts or engaging in a variety of experi- 
ences, (2) the environment as providing a structure that permits 
learning and negotiating with adults who are responsible for 
guiding the learning, and (3) the environment as providing 
gratification (feeling of self-confidence) for initiating and 
carrying out work. Environmental complexity is suggested when 
higher levels of the factors are perceived as being present in 
the environment. That is, an environment that is perceived as 
having more opportunities for generating concepts is more complex 
than one that is perceived as having fewer opportunities. 
When we apply this interpretation of the LEQ to the two groups of 
students in this study, the learning-in-work students when compared 
to the traditional learnina students perceived their environment 
as providing (1) greater opportunity to generate concepts, U) more 
structure for negotiating work, and (3) more gratification for 
initiating and completing work. Thus, according to this measure 
(LEO) and the current sample of students, the learning-m-work 
students appeared to perceive a more complex environment for 
learnina than did the traditional learnina students. 
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Cross-Lagged Analysis of Math Scores Based on 



Predictions from Bloom's Taxonomy 



Introduction 

The intent of the second area of inquiry was to explore further the 
unusual patterns of .performance exhibited on the mathematics scales 
of the CTBS, As illustrated -in the preliminary report, the pat- 
terns of learning/retention, when broken down by program, suggested 
that different processes were involved for the tv/o different 
learning environments . 

One technique for assessing the validity of the above hypothe- 
sis of different causal processes in the group is- cross-lagged 
panel correlation (CLPC). For the purposes of our analyses of 
the CTBS math performance , CLPC techniques were used according to 
the procedures outlined by Kenny (1975, 1979), Briefly, CLPC is 
a quasi-experimental design that attempts to rule out alternative 
hypotheses to a causal effect between two variables- The primary 
alternative hypothesis in these cases is that of spuriousness : 
i.e., one variable does not cause the other; instead, they are 
related by virtue of a common relationships to a third (unmeasured) 
variable. For example, the correlation between height and math 
achievement may not be due to a causal relationship between the 
two; instead, they could be related because increases in both are 
caused by maturation. The usefulness of CLPC lies in its ability 
to provide support for the existence (and form) of a causal 
relationship between variables by ruling out the alternative 
hypothesis of spuriousness. 

CLPC assesses the presence/absence of a true causal effect 
through an examination of the correlations among variables 
measured at two points in time. Since CLPC has a somewhat non- 
standard terminology associated v/ith it, t:he following section 
will define a few key terms. Basically there are three types of 
correlations in CLPC: autocorrelations; synchronous, and cross- 
lags. 

Given two variables; A and B, each measured at two time 
periods, 1 and 2, autocorrelation is defined as the correlation 
of a given variable with itself, measured at two different time 
periods. Accordingly, there are two autocorrelations in our 
example: correlation of A at time 1 with A at time 2, and B at 
time 1 with B at time 2. Synchronous correlations are defined as 
correlations among different variables measured at the same time. 
Therefore, there are two synchronous corre_lations in our example: 
A at time 1, with B at time 1, and A with time 2 with B at time-2. 
Finally, crosslag correlations are defined as correlations of a 
given variable, at one time period, with a second variable mea- 
sured at another time period. So, there are two crosslags in this 
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sample: A at time 1 with B at time 2, and B at time 1 with A at 
time 2. Given the correctness of the assumptions of synchronicity 
(the variables are measured at the same times at each wave) and 
stationarity (that the causal processes, or structural equations 
of each variable, remain constant or proportional over the obser- 
vation time) , the null hypothesis of CLPC is that the cross-lagged 
correlations (of the variable A at time 1 with variable B at time 2, 
and vice versa) are equal. Equality of the two crosslags is seen 
to indicate a spurious relationship between the two variables. 

Significantly different crosslags, given the validity of the 
synchronicitv and stationarity assumptions, are seen to indicate 
the presence' and form of a causal effect between the variables. _ 
With respect to the satisfaction of the assumptions, synchronicity 
can be assured if the measures were gathered at the same time for 
each wave, not aggregated over time. Stationarity, on the other 
hand, can be evaluated by examination of corrected synchronous 
correlations, corrected by the process outlined in Kenny (1975). 
For these analyses, the model of stationarity assumed was quasi- 
stationarity , which implies that the synchronous correlations 
will be equal when corrected for shifts in reliability (or, more 
correctly, communality) over the observation periods. 

For the purposes of these analyses, the CLPC techniques will 
be used as suggested by Kenny (1979), in that significant crosslag 
differences will be seen only as indicators of a potential causal - 
relationship between the variables, and not as "proof" of causation. 
The basic strategy in this section is to compare CLPC re:=5ults _ 
between the two groups for corresponding time lags, using variable 
time lags on all subjects with complete data at these points. This 
use of subjects is different from that used in the preliminary _ 
report, which used only students with complete data at all adminis- 
trations; however, the current approach was taken in order to use 
the strategy of Kenny (1979) in replicating the analyses on as 
many as possible combinations of subjects and tine lags in order 
to increase confidence in the findings. While the experimenters 
are well aware of the limitations and potential instability of the 
parameters of the CLPC models obtained from small samples (like 
those used here), these replications of significant results, when 
viewed in combination with other analyses on these data, may 
then suggest profitable future areas for more thorough research. 
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Results 



The CLPC analyses were conducted separately for the treatment 
(learning- in-work environment) and control (traditional learning 
environment) students • First, correlation matrices were completed 
for both groups on all subjects present at the following time 
periods: pre-test and post-test; pre-test and follow-up; and 
post-test and follow-up. Since several students had at least one 
set of missing data, this procedure resulted in the maximum sample 
size for each period, which in turn contributed to an increase in 
power for the Z_ test of the crosslag difference. 

The correlations were computed on four variables constructed 
from the two CTBS math scales according to instructions in the 
CTBS test manual. These scales were designed to measure cognitive 
properties/processes, derived from Bloom's Taxonomy of Educational 
Objectives , in the following areas: Recognition, Translation, 
Interpretation, and Analysis. More information on the items used 
in these scales was presented in chapter II. 

Results of the individual CLPC analyses are presented in 
Appendix C and are summarized in figure 13 • 

As a means to validate the CTBS qroupinq of items to form the 
scales to measure concepts derived from Bloom's taxonomy, a factor 
analysis was performed on CTBS items 1-50 at pre-test on all sub- 
jects. The analysis employed the same procedures used in the 
factor analysis of the LEQ described earlier. Results of the Pro- 
max transformation of the three factor solution are presented in 
Table 6. Criteria for retention of factors were (1) parallel 
analysis results, which indicated three factors with eigenvalues 
greater than those expected from random data and (2) examination 
of discontinuity in the eigenvalues, which showed a sharp break 
after the third factor. Other decision rules (e.g., retaining 
eigenvalues greater than one) were rejected as both arbitrary and 
inappropriate for these data. 

The results of the factor analysis, while clearly not consist- 
ent with the grouping of items as performed by CTBS, were deemed 
useful in allowing another, more empirically-based, summation 
strategy to form research variables for use in CLPC. Starred item 
loadings in Table 6 indicate the items used to form each of the 
three CTBS factor-based scales. These scales, like the scales 
formed from the LEQ items earlier, are not based on factor scores; 
instead, the present strategy was to use the sum of unit-weigntea 
(1 or -1, depending on the sign of the loading) CTBS items corre- 
sponding to non-trivial (over .30) loadings on the oblique factor 
solution. Means and other summary statistics are reported for 
these variables in Table 7. 

The results of the individual CLPC analyses on the factor- 
based CTBS scales are presented in Appendix D and a summary of 
the findings is depicted in figure 14. 
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FIGURE 13 

Summary CLPC Results for 
CTBS-Bloom Scalesl 

It Post-Test Follow-up 



Learning Interval Retention Interval 




R 
T 

I 
A 



R 
T 

I 
A 



^Solid line indicates significant differences, at p-<.10. Dotted 
lines indicate differences at p<.20. 
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TABLE 6^ 



Promax- Rotated CTBS Math Factor Solution 

PRIMARY FACTOR PATTERN 



RESULTS OF PRESENT ANALYSIS 



CTBS/BLOOM'S SCALES 



ITEM 





I 


II 


III 




T 


T 


A 


1 


.50* 


.23 


.00 






* 




2 


.25 


.31* 


.23 


* 








3 


-.04 


.15 


.38* 


* 








4 


.06 


.51* 


-.20 






* 




5 


-.04 


.60* 


.20 


* 








£ 

D 


— , 01 


. 51* 


.33* 










7 


-.03 


.14 


.56* 


* 








8 


-.06 


.23 


.11 






* 




9 


^.15 


.13 


.34* 






* 




10 


.32* 


.06 


.15 










11 


.15 


. 08 


.15 








* 


12 


,44* 


.07 


-.14 






* 




13 


.51* 


.15 


-.05 








* 


14 


. 02 


.06 


.60* 








* 


15 


.23 


.23 


-.15 








* 


XD 


1 Q 


.4 4 * 


.14 






* 




17 


.04 


.04 


.57* 








* 


18 


.10 


-.10 


.42* 






* 




19 


-.19 


.38* 


.59* 






* 




20 


. 20 


.33* 


.11 










21 


.42* 


. 33* 


— .10 








* 


22 


.40* 


. .02 


.24 






* 




23 


.10 


. 01 


.30'^ 






* 




24 


. 21 


.36* 


-.14 






* 




25 


.28 


.59* 


-.11 








* 


26 


.18 


.60* 


.14 










27 


.35* 


.45* 


.08 






* 




28 


. 09 


.28 


.26 






* 




.29 


.29 


.05 


.24 






* 




30 


.00 


.07 


.00 






* 




31 


.09 


.09 


.28 






* 




32 


-.10 


.43* 


.29 






* 




33 


-.09 


.11 


.50* 




* 






34 


.48* 


.24 


.13 






* 




35 


.6 5* 


.09 


.06 




* 






36 


.39* 


.14 


«.14 






* 




J / 


.fly * 




no 










38 


.42* 


.06 


.02 






* 




39 


.23 


.00 


.40* 






* 




40 


.29 


-.06 


.29 






* 




41 


.66* 


.07 


.10 










42 


.42* 


. .12 


.12 








* 


43 


.06 


-.22 


.54* 






* 




44 


.46* 


-.12 


.32 






* 




45 


.48* 


-.12 


-. 06 








* 


46 


.72* 


-.19 


.06 








. * . . 


47 


.36* 


.01 


.10 






* 




48 


.42* 


-.12 


.14 








* 


49 


.33* 


-.29 


.13 






* 




50 


.37* 


-.32* 


.45* 






* 





'Indicates items associated with the factor or scale 
N - 80 



Factor Correlations 



II 



III 



I 

II 
III 



1.00 .35 .44 
.35 1.00 .33 
.44 .33 1.00 
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TABLE 7 



Summary Statistics for CTBS-Factor Scales- 



Pre-Test (N = 80) 
I 

II . 



CTBS 
CTBS 
CTBS III 



post-Test (N = 94) 
CTBS I 
' CTBS II ^ 
CTBS III 



Follow- up (N = 66) 
CTBS I 
CTBS II 
CTBS III 



Mean 
0.621 
0.769 
0.571 



0.629 
0.759 
0.574 

0.633 
0.791 
0.622 



Standard Deviation 
0.254 
0.227 
0.265 

0.253 
0.244 
0.264 

0.275 
0.217 
0.250 



1 Range is from 0 to 1, where. 1 = correct, 0 = incorrect 
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FIGURE 14 



Summary CLPC Results for 
CTBS Factor-based Scalesl 



Pre~Test 



Post-Test 



Follow-up 



Learning- 
Students 



Learning Interval Retention Interval 
I- ~ ^ _ , -I ' I 

III-^--^^^^ -III 




I 
II 
III 



I 
II 
III 



1 

Solid lines indicate p-<.10 crosslag differences; dotted 
lines indicate p-<.20 differences'. 
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Interpretation 

The present interpretations and uses of CLPC are somev^hat 
atyoical, in that CLPC was employed to estimate the parameters of 
the" CLPC model for separate groups of subjects. These models were 
then examined subjectively to detect patterns in the significant 
crosslags for each group that would assist us in evaluating the pre- 
sumed dissimilar causal processes in the two groups. However, the 
purpose of these analyses is not to "prove" the existence of 
specific causal relationship in the CTBS scales/concepts; rather 
it is to discover significant and also marginally significant 
differences (given the low power of the Pearson-Filon Z test) in 
the crosslags or nonsymmetric patterns (i.e., for treatment vs. 
control) . These differences or patterns would be seen as offering 
support (1) to. the hypothesis that different causal processes 
were operative in, these two learning environments, insofar as the 
variables represented by our observed variables are concerned arid 
(2) to the initiation of controlled, systematic investigations into 
the effects of different environmental conditions on the acquisi- 
tion of knowledge and academic performance, v/here stronger 
inference 3 of causality, seem possible. 

As a consequence of this "exploratory" use of CLPC, interpre- 
tations of the results summarized in figures 25 and 32 will be 
restricted to specific differences in sign of the significant 
crossLag differentials. It is asserted here that these sign 
differences reflect, at the least, differences in the relationships 
among the variables measured in our sample (i.e., solely descriptive 
of different structure) . At best, these differences may suggest 
true differences which may be a function of the different 
structuring of the two learning environments. 

Inspection of the CLPC results revealed no consistent patterns 
of suspected causal relationships between the two groups. With 
reqard to the predictions generated from Bloom's taxonomy, the 
signs of the crosslag differentials (on significant pairs) were 
not consistent with the notion that lower-order concepts should 
cause higher-level concepts. That is, if R was seen as causing 
an increase in T, then the correlation of R at time 1 with T at 
time 2 miH^Ts the correlations of R at time 2 with T at time 1 
should be a positive value. 

To illustrate. Figure 14 shows the crosslag differential _ 
between the R and A variables to be significant (p= .06). This 
differential is composed of the difference: ^i^o ".^2^1' °^ . ^ 
.45 - .68, or -.23. This would be consistent with increases m R 
(recognition) causing a decrease in levels of A (analysis, a 
higher-level concept) at time 2. 

Overall, the results of the CLPC analyses indicated (l) the 
presence of several cases (pairs of variables) for which the null 
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hypotheses of no causal relationship (i.e., spuriousness) was 
deemed unacceptable and (2) an asymmetric structure of those 
cases, such that both their locations and effect-signs (crosslag 
differentials) were different for treatment versus control student 
samples. Little support was seen for the predictions of hier- 
archical relationships for the CTBS/Bloom's taxonomy scales or 
for the same, processes operating in both environments. 
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Effects of Placement Experiences on Performance 

Results 

In the course of their program activities, the students in 
the learnina-in-work group chose placements at various sites in 
the communitv, v/here their "hands on" experiences took place. 
It was of interest in the present analyses to determine whether 
differences occurred in math performance of students as a function 
of differences in site placements. Specifically, were'there 
differences between students who had at least one exposure to 
one of the twelve worker-trait-group-classified job clusters, 
and those who did not have exposure to that cluster? If these 
clusters were valid for the purpose of classifying the charac- 
teristics of a given site and if some of these characteristics 
also had an impact on the acquisition of math skills, then it 
would be expected that there would be differences (on post-test 
and/or follow-up CTBS scores) between participants and non- 
participants in those worker trait groups (WTG) v/hich had an 
impact on math- skill acquisition. >7hile this type of analysis, 
could certainly not address the issue of which caused the other 
(i.e., did the T'^TG classification experience cause later math 
achievement) , the discovery of differences as a function of 
placement 'would tend to contribute support to the EBCE hypothesis 
that the type of site characteristics has an impact on what is 
learned at" the site. In addition to the fact that the students 
self-selected themselves into the. sites, the presence of differences 
could also be due to an interactive effect of the self-selection 
and the characteristics of the site. This alternative explana- 
tion, however, could not be addressed in this analysis. 

It was also of interest to examine the importance of the 
number of discrete site experiences in explaining the math 
performance variance. That is, does knowledge of the number of 
different sites a student was exposed to during the academic 
year help to account for the differences in CTBS performance? 

The breakdowns of participation in each of the V7TG classified 
sites is presented in Tafcle 8. Examination of Table 8 shows that 
several of the V^TG classifications had very few students partici- 
pants (e.g., WTG 2, 4, 6, 9, and 12 had fewer than 25 percent 
of ■ students taking even one experience in the group) . Because 
of the small sample size (since only learning-in-work students 
TDarticipated) and the large number of predictor variables, only 
WTG groups 1, 3, 5, 7, 8, 10, and 11 were used in the analysis m 
the interest of maximizing the power of the statistical tests. 
Results of simultaneous regression analyses using the two overall 
CTBS math scales as dependent variables are reported in Table 9. 
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Results of the correlations of the .number of sites with' the 
same CTBS scales used above are reported in Table 10. 



Interpretation 

While differences between the participants and non-partici- 
pants are apparent at post-test for WTG classifications 1 and 11 
in concepts^ and WTG 11 and application (all p's^.lO), which 
would tend to suggest an effect for these classifications, 
inspection of the results for these same students at pre-test 
(before exposure to the sites) reveals that these same variables 
showed tendencies toward the same differences seen at post-test. 
For example, although students with exposure to sites classified 
under WTG 11 showed a 5.06 point post-test (in CTBS units) 
superiority (on the average) to students without such experience 
(holding the other variables constant) , these same students were 
3.64 units higher on the average at pre-test also. No comparisons 
approached significance at the following period . 

The presence of these initial differences before on-site 
experience casts strong doubt on the efficacy of the worker 
trait classifications as causal agents in this sample . 
As a result of the type of analysis used here (simultaneous 
analysis, using dummy coded variables) , these findings represent 
mean differences while holding the effects of the other WTG 
variables constant; this strategy was deemed appropriate for this 
analysis, since the significance of the overall multiple corre- 
lation was not a prerequisite for the interpretation of 
the regression coefficients- While separate F-tests of each of 
the WTG groupings would have been expected to shovr more "signifi- 
cant" differences between students on the basis of participation 
in each WTG, this strategy was rejected since it did not control 
for the effects of the other WTG groupings. 
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TABLE 8 



Participation in WTG Sites 

Frequency 



WTG 1 - Artistic 
No 
Yes 

WTG 2- Scientific 
No 
Yes 



22 
16 

31 
7 



Percent 

58 
42 

82 
18 



WTG 3 - 


Nature 




63 




No 
Yes 


24 

1 A 


WTG 4 - 


Authority 

No 

Yes 


30 

Q 
O 


79 

Z ± 


WTG 5 - 


Mechanical 




40 
o u 




NO 
Yes 


15 
^ J 


WTG 6 - 


Industrial 


36 


95 

c 
D 




No 
Yes 


WTG 7 - 


Business 


• 

26 


68 


• 


No 
Yes 


WTG 8 - 


Sales 


14 
24 


37 
63 




No 
Yes 


WTG 9 - 


Acconunoclating 

No 

Yes 


31 
7 


82 
18 


WTG 10 - 


Humanitarian 


17 
21 


45 
55 




No 
Yes 


WTG 11 - 


Social 


14 
24 


37 
63 




No 
Yes 


WTG 12 - 


Physical Performing 


36 
2 


95 
5 




No 
Yes 



n = 38 
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Results of Regression Analysis of WTG Placements 



Pre-Test 



D.V. = CTBS 



D.V. = CTBS 



Post-Test 





Parameter 


t 


p 


- Concepts 








WTG 1 


2 .22 


1.16 


.26 


WTG 3 


1.45 


0.76 


.45 


WTG 5 


-1.42 


-0 . 53 


. 60 


WTG 7 


1.06 


0.49 


.63 


WTG 8 


1.24 


0 . 61 


.54 


■ WTG 10 


-0.54 


-0 .21 


. 83 


WTG 11 


3.64 


1 . 68 


.10 


- Applications 






WTG 1 


-0.38 


-0.16 


.87 


WTG 3 


-0.61 


-0.26 


.80 


WTG 5 


-0.87 




RO 

m O \J 


WTG 7 


-0.09 


-0.03 


.97 


WTG 8 


1.06 


0.43 


.67 


WTG 10 


0.57 


0.19 


.85 


WTG 11 


3.76 


1.43 


.16 




Parameter 


t 


£ 



R = .18 



R = .11 



D.V. = CTBS - concepts 



WTG 


1 


3 .57 


1.88 


.07 


WTG 


3 


1.96 


1 .04 


.31 


WTG 


5 


-0.85 


-0.32 


.75 


WTG 


7 


1.83 


0.85 


.40 


WTG 


8 


0.68 


0.34 


.74 


WTG 


10 


-1.00 


-0.40 


.69 


WTG 


11 


5 . 06 


2.36 


.02 


D.V.= CTBS - Application 








WTG 


1 


,0.45 " 


0.21 


.83 


WTG 


3 


1 .76 


0.83 


.41 


WTG 


5 


3.81 


1.37 


.18 


WTG 


7 


0.24 


0.10 


.92 


WTG 


8 


-1.22 


-0.53 


.59 


WTG 


10 


0.42 


0.16 


.87 


WTG 


11 


4 .77 


2.09 


. .05 



N = 38 



R = .28 



R = .20 
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TABLE 9 (continued) 



Results of Regression Analysis of WTG Placements 
Follow-up 







Parameter 


t 


D.V. = CTBS 


- Concepts 








WTG 1 


1 . 75 


u . / ^ 




WTG 3 


1.60 


0.65 




WTG 5 


-2 . 22 


-0 . by 




WTG 7 - 


0.52 


0 . 20 




WTG 8 


3 .19 


1. 24 




VfTG 10 


-2.09 






WrG 11 


2. 31 


0 . 74 


D.V. = CTBS 


- Applicat 


ion 






WTG 1 


0.72 


0.25 




WTG 3 


1.80 


U • o u 




WTG 5 


-4.66 


-1.20 




WTG 7 


-3.02 


-0.99 




WTG 8 


3.67 


1.18 




WTG 10 


-3.17 


-0.80 




WTG 11 


1.67 


0.44 


n = 30 









.48 
.52 
.49 
.84 
.22 
.52 
.47 



.80 
.55 
.24 
.33 
.25 
.43 
.66 



= .13 



R = .13 
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TABLE 10 

Squared Correlations of Number of Site Experiences 

with Math Scores 

CTBS - Concepts CTBS - Application 

Pre-Test (N = 38) .021 .011 

Post-Test (N = 38) .068-'" .055^ 

Fol low-Up (N = 30) .069-^ .009 
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Effects of Summer Activities on Performance 



Results 

The research question addressed by this section concerned 
student activities during the summer break (i.e,, between post- 
and follow- up test) and their association v/ith math achievement. 
Specifically, these analyses addressed the issue of whether 
distinctions (1) between summer school or no summer school, (2) 
between work or no work/ and (3) between treatment or control 
would account for significant amounts of CTBS score variance. 

An alternative explanation of the increase in scores observed 
following the summer recess could be that this increase was an 
artifact, resulting from greater exposure to learning situations 
(i.e., summer school). If the students had worked during the 
summer, a similar increase in performance might have been expected 
due to the opportunity to practice skills learned .during the 
school year (provided that such an opportunity existed) . Another 
alternative explanation to significant differences between student 
on the summer-work and summer-school variables could be that these 
differences represented pre-existing differences; thus, a hypothe- 
tical finding that students in summer school scored higher at the 
follow-up test could be due to their superiority at pre- and post- 
test, rather than to the extra classwork. To address these issues 
simultaneous regression analyses were conducted at all testing 
periods. The effect of summer school attendance and a summer job 
was examined on CTBS m.ath performance, while the effect of the 
program variable vzas held constant (i.e., partialled from each). 
The result of these analyses are presented in Table 11. 

'Table 12 presents means for the four groups formed by the 
combinations of summer school (yes or no) and summer job (yes or 
no) for the total sample of learning-in-work group only-by-the 
control students only. 



Interpretation 

As is evident in Table 11, summer activities did not make 
much of a contribution to explaining CTBS score variance, inde- 
pendent of program effects: none of the multiple correlations^ 
were significant at p = .05, and only summer school participation 
showed strong effects (on math content, at post-test and follow- 
up) . The general pattern was that students who went to summer 
school during the break showed lower mean math scores at all 
testings, while students who worked showed generally higher mean 
scores. These results suggest non-comparability of students in 
that there are rel^ively consistent trends for differences in 
students when classaTf^ied by summer activity. 



TABLE 11 

Results of Analysis of Summer Activities 



CTBS - Math Content 



Pre-Test 

Intercept ^ 
Sxaminer School 
Work 2 
Program 

Post-Test 

Intercept 
Summer School 
Work 
. Program 

Follow-up 

Intercept 
Summer School 
Work 
Program 



Parameter 

17.27 
-2 .74 
1 .53 
-0.78 



18.28 
-4.19 
1.32 
-1.97 



18 .19 
-4.26 
1.68 
-1.32 



•1.28 
0.88 
■0.61 



-2 . 05** 
0.75 
-1 .51 



■2 . 00** 
0.92 
-0.97 



R = .04 



R = .11" 



R = .09 



CTBS - Math Application 

Pre-Test 

Intercept 
Summer School 
Work 
Program 

Post-Test 

Intercept 
Summer School 
Work 
Program 



15.91 
-1.05 
-0.86 
-0.69 



16.68 
-3.17 
1.17 
-2 .69 



•0.41 
•0.42 
■0.45 



•1.34 
0.58 
■1.82 * 



R = .01 



R = .10 



Follow- Up 

Intercept 16.28 

Slammer School -2.12 -0.80 

Work 0.39 0.17 2 

Program -1.97 -1.16 R = .04 

Summer school participation coded 1; otherwise, 



n = 54 
* p-c.lO 
**p-< . 05 



1. 
2. 
3 . 



Work participation coded 1; otherwise, 0 
Program coded 1 if learning-in-work students; 
otherwise, 0 
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TABLE 12 

CTBS Mean Scores by Summer Jobs and Summer School 





T o u a. X 


1 X uinen u 


L.X UXO 










Pre-Test 








SS* ancl VJork 


15.7 


15.2 


16.0 




14 .1 


13 .7 


14.5 


Work only 


18.4 


18 .0 


18 .8 


None 


16 .8 


16 .4 


17 . 2 


Post-Test 








SS and VIork 


14.3 


13 .4 


15 .4 




12 . 8 


12 .1 


14 . 0 


Work only 


18.6 


17 .6 


19.6 


None 




XD • O 


18 . 2 


r (J X X w u.^ 










14 . 9 


14 . 2 


15.6 


OO iJilXjf 


13 . 0 


12 .6 


13.9 


Work only 


19.2 


18 .5 


19.8 


None 


17 .4 


16.8 


18 .1 


V^Xi30 X X ^ d UXUll 








Pre-Test 








SS and Work 


13 .7 


13 .3 


14 . 0 


o o J 1 X y 


14 . 4 


14 .1 


14 .8 


Work only 


14.7 


14 .3 


15.0 


None 


15.5 


15.2 


15.9 


Post-a'est 








SS and V^fork 


13.4 


11.9 


14 . 6 


SS only 


11.7 


10.8 


13.5 


Work only 


16 .6 


15.1 


17 .8 


None 


14 .9 


13 .9 


16 .6 


Follow-up 






14.5 


SS and Work 


13 .4 


12.8 


SS only 


12 .8 


12 .1 


14.1 


Work only 


15.7 


14.7 


16.6 


None 


15.1 


14 .3 


16.2 


*SS = Summer School 


n = 54 


n = 27 


n = 27 
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Relationships between Student's Perceptions of the 



Learning Environment and Cognitive Style on Performance 



Results 

This section of the analysis is concerned with the form of 
the person-environment relationship. For this study, the personal 
characteristic of cognitive style was measured by the paragraph 
completion test (PCT) , and the student's perception of the com- 
plexity of the environment was measured by the LEO. The pot Ig 
a content-free measure of cognitive style^f^integrative com- 
plexity. It measures an individual's ability to aenerate irlp^as^ 
or to differentiate among alternative solutions. It also is a 
measure of the person's ability to orqanize ideas or concepts in 
a hierarchical system. Table 13 displays the means and standard 
deviations for the students' cognitive style scores. 



TABLE 13 

PCT Means and Standard Deviations 

EBCE (n = 52) 

Mean 2,33 
Standard Deviation 0.82 

control (n = 41) 

Mean 2 . 15 

Standard Deviation 0.75 

Total (n = 93) 

Mean 2.25 
Standard Deviation 0.79 

The position of the authors is that the relationship of per- 
son, environment, and performance (as measured by the^CTBS) is 
inter active , because the relationship of the structure of the 
learning environment to performance in that environment will vary 
as a function of the characteristics of the person (the 
cognitive style) . In this sample, optimum performance is pre- 
dicted for students who perceive a more complex environment, as 
indicated by higher LEQ scores, and who exhibit greater integrative 
complexity as measured by the PCT. Thus, when the level of environ 
mental complexity is controlled for, the performance of students 
is predicted to be uniformly hiqher a^s a function of increased 
integrative complexity or cognitive style- Accordingly, a 
significant "main effect" for the cognitive style variable is 
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oredicted-, together with a significant interaction effect when 
performance is predicted from perceived environmental complexity 
and cognitive style. Overall, the hypothesis is that performance 
is maximized v/hen the students ' . level of cognitive style is 
matched with similar levels of perceived environmental complexity 
and that performance is minimized when there is a mismatch 
between the two variables (e.g., when students with less integrative 
"c^plexity are placed in an environment perceived as complex) . 

To assess the predictions regarding the environmental and 
cognitive complexity interaction, hierarchical regression analyses 
were used as follows. First, the variables representing the PCT 
and LEQ were used as predictors of CTBS math performance (scales 
for Application and Content) . Then, the set of product terms 
representing the interactions (i.e., PCT x LEQI, PCT x LEQII, and 
PCT X LEQIII) was entered, and the squared multiple semipartial 
correlation for the interaction used as a measure of the contri- 
bution of these terms. While the membership in' either of the two 
environments could conceivably have been used as an "objective" 
measure of environmental complexity, this approach was rejected 
at this stage, since it was felt that individual differences in 
perceptions would constitute more meaningful information than the 
"program" variable. The results of the hierarchical regressions 
are reported in Table 14. 

These results indicated that no interactive relationship was 
present for the cognitive style and environmental complexity 
variables, as seen by the non-significant F values for the 
increment in r2 with the addition of the interaction terms. How- 
ever, the prediction of a "main effect" for the cognitive style 
variable was supported (using the model without the interaction 
terms) as seen in Table 15. Specifically, higher scores on the 
PCT were associated with higher achievement (as measured by the 
CTBS) across all of the levels of perceived environmental complex- 
ity (as seen in the significant t values for PCT at both post-test 
and follow-up. A smaller main effect was seen for the LEQ/ 
environmental characteristics set, which was strongest at post- 
test. Although weak relationships were observed, generally an 
increase in the LEQ factors I and III produced a small decrease 
in math scores at both post- and follow-up testings. Increases 
in LEQ factor II showed a decrease in math content scores at 
post- and follow-up testings and an increase in math applications 
scores at both testings • 

In an attempt to discover the person-by-environment 
interaction through a different conceptualization of the 
"environment" construct, variables representing other aspects 
of the environment were added to the model: (1) summer activities, 
consisting of whether the student attended summer school during 
the junior-senior break and whether the student was employed 
in paid employment during that period; and (2) academic 
program, whether the student was enrolled in learning- in-work 
or "traditional" curriculum structure. It was reasoned that 
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the inclusion of -the above variables would perhaps provide a more 
complete representation of salient environmental characteristics, 
and therefore a more accurate test of the person x environment 
hypothesis . 

The amalyses reported in Table 14 were replicated with the 
addition of summer activity variables to the set of variables 
representing the environmental characteristics factor. These 
results are surjimarized in Table 16. 

The results of using summer activities and the program clas- 
sification were the same as found for using the LEQ scales alone 
in that the non-interactive variance and the interaction of PCT 
by environment produced no significant increase in explained vari- 
ance beyond the n on- interactive model. 

Interpretation 

When we noted the parallel nature of the above findings with 
respect to an interactive relationship between personal character- 
istics and environmental characteristics, it was concluded that 
no support for this interactive hypothesis was provided. How- 
ever, as seen in Table 15, the prediction of a "main effect" 
for PCT was supported, since higher scores on this measure of 
cognitive complexity- were associated with higher scores on the 
CTBS scales (controlling for the effects of different perceptions 
of the complexity of the environment) . Additionally, some 
indication was given to a "main effect" for the LEQ scales, 
though this was not of s;..ch magnitude as the PCT effect. 
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TABLE 14 
Tests of Interactions 



Post-Test 



CTBS - Math Content (n = 82) 



. ^ ^ Cuntulative Incremental 
Varxables added to 2 — 2 

equation R 1] 

9r5r* — 

PCT, LEQI-LEQIII .205 4.97 

PCT X LEQI, PCT X LEQII .215 .010 0.32 



PCT X LEQIII 
CTBS - Math Application 



PCT, LEQI-LEQIII .12 7 2.62 

PCT X LEQI, PCT X LEQII, .131 .004 0.11 
. PCT X LEQIII 



Follow-up 

CTBS - Math Content (n = 59) 



CTBS - Math Application 



icic ic 

PCT, LEQI-LEQIII .260 4.75 

PCT X LEQI, PCT X LEQII, .265 .005 0.12 
PCT X LEQIII 



PCT, LEQI-LEQIII .143 2.24 

PCT X LEQI, PCT X LEQII, .149 .006 0.12 
PCT X LEQIII 



* 



* p< .10 
** p< .05 
*** p-c.Ol 
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TABLE 15 

Partial Coefficients of Noninteractive Model 



Post-Test (n = 82) 
CTBS - Math Content 



Intercept 

PCT 

LEQI 

LEQII 

LEQIII 

CTBS - Math Application 

Intercept 

PCT 

LEQI 

LEQII 

LEQIII 



Raw Score 

Regression 

Coefficient 



16.73 
3.02 
-0.82 
-0.46 
-0 .26 



13.77 
2 .38 

-1.48 
1.26 

-0.53 



Squared 

Semipartial 

Correlation 



194 
030 



.107 
.036 



4 . 34*** 
-0.89 

-0.47 2.91** 
-0-28 



2 . 97*** 

-1.38 
1. 02 
-0.46 



3.18** 



?ollow-up (n = 59) 

CTBS - Math Content 

Intercept 

PCT 

LEQI 

LEQII 

LEQIII 

CTBS - Math Application 

Intercept 

PCT 

LEQI 

LEQII 

LEQIII 



17 .03 
3.19 
-1.53 
-0.09 
-0.10 



12 .99 
2.92 

-1.29 
0.61 

-0.39 



.239 
.036 



.137 
.017 



4.18*** 
•1.34 

■0.08 2.63* 
0 .11 



2 .93*** 
-0.87 

0.42 1.07 
0.30 



* p-c.lO 
** p<.05 
*** p<.01 
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TABLE 16 



Tests of Interactions"*' 



Cvnnulativ^e . F for 

^ 



Pre-Test 



Content 



Increment 



PCT^ LEQ^ SS,Work, Program .209 1.93 

+ Interaction .32 0 .111 1.22 

Application 

PCT, LEQ, SS/ Work, Program 223 2.09 

+ Interaction -276 ^053 0.55 

Post-Test 
Content 

PCT, LEQ, SS, Work, Program .357 4.04** 

+ Interaction .4 90 .133 1.96 

Ar>TDlication 

PCT, LEQ, SS, Work, Program .289 2.96** 

+ Interaction -351 .062 0.72 

Follow-up 
Content 

PCT, LEQ, SS , Work, Program .347 3.87** 

+ Interaction ^ .482 .135 1.96 

Aoplicatio^x 

PCT+LEQ, SS , Work, Program .195 1.76 

+ Interaction .28 8 .093 0.98 



*** p -c.Ol 
** p ^. 05 n = 59 

"^Interaction refers to the interactions of the personal characteris- 
tic of cognitive style with the set of environmental characteristics 
of Work, SS, LEQ and Program. 
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Discussion of Results 



In chapter I, (figure 4) the authors postulated an heuristic 
model for investigating student cognitive processes in learning- 
in-work environments- The discussion of the results of the 
hypothesized causal relationships reflects the structure of that 
model. 

First, did being in one of the two learning environments 
cause students during the learning or retention intervals to- learn 
math skills as predicted by Bloom's taxonomy and as measured by 
the CTBS? The C^CBS process scales of Recognition (R) , Translation 
(T), Interpretation (I), and Analysis (A) were hierarchically 
developed along the lines of Bloom's Taxonomy; therefore, it was 
hypothesized that students would learn the concepts in such a way 
that, for example, a mastery of the cognitive skill of Recognition 
at pre-test would increase the students' performance in the skill 
of Translation at the post-test. The predicted and actual results 
of learning the cognitive skills is depicted in Fignare 33. 



FIGURE 15 



Predicted and Actual Causal Relationships 
between Cognitive Concepts for Students 
in the Two Learning Environments 



Learning- 
In- 
Work 
Students 



Conceptual 
Time 1 Time 2 




Predicted 



Time 1 Time 2 



T ^'^^^^^ 




Actual 

Learning Interval 
Pre Post 




Acliual 

Retention Interval 
Post Follow-up 




Tradi- 
tional 
Learning 
Students 





R 



Keys 



= Causes 
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The results suggested that the cognitive concepts were 
causally related but not in the hierarchical order predicted by 
Bloom's taxpnomy as interpreted and measured by CTBS. Furthermore, 
the results would indicate that different causal processes were 
operating in the two environments as manifested by the number 

and type ..of ^...rejations^^^ The,J.earning_,i^ f qr_ the learning- 

in~work students shov/ed the greatest number of potential causal 
relationships. During the retention interval both groups showed 
relationships between two variables, but for different cognitive 
concepts. Overall, these results are consistent with the hypothesis 
that there are different causal processes operating for learning 
math concepts as a function of being in one of the two learning 
environments . 

Secondly, for the learning~in-work students, did exposure to 
worksites, either by the number engaged in or by the WTG classifi- 
cation, affect math performance during the learning or retention 
interval? While differences between participants and non-partici- 
pants are apparent at post-test for WTG classifications .1 (Artistic) 
and 11 (Social) on the math scale of concepts and WTG 11 on the 
math scale of applications, inspection of the results for the same 
students as at pre-test reveals that their same variables showed 
patterns toward the same differences observed at post-test. Thus, 
the presence of these initial differences before the experience 
casts strong doubt on the efficacy of the WTG classification as 
causal agents in this sample. That is, exposure to sites as 
classified by WTG was not related to math achievement. 

Thirdly, did the students' summer activities affect math 
performance during the retention interval? Generally, summer 
activities did not contribute much to explain math performance at 
the follow-up testing. However, the general pattern was that 
students who attended summer school showed lower mean math scores 
at all testings and had the greatest gains over the summer, while 
students who worked during the retention interval tended to perform 
higher on the math scales. 

Finally, is there a person-by-environment interaction for 
ma*:h performance for either the learning or retention interval? 
Using the LEQ as a measure of perceived environmental complexity 
and the PCT as a measure of cognitive style,\ we observed a main 
effect for both variables. That is, higher cognitive style scores 
were associated with higher achievement for both the learning and 
retention intervals across all levels of perceived environmental 
complexity. Although v/eak, a reverse relationship was generally 
observed for the perceived environmental complexity in that an - 
increase in LEQ scores was associated with small decreases in 
math performance. The addition of other measures of environmental 
characteristics (i.e., learning-in-work vs. traditional classroom, 
type of summer activity chosen) did not change these findings of 
no interaction. Overall, the cognitive style variable exhibited 
uniform effects on math performance, the environmental perceptions 
(LEQ) showed a smaller, less consistent effect, and the predicted 
person-by-environment interaction failed to materialize. 
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CHAPTER TV 



IMPLICATIONS AND RECOMMENDATIOHS FOR 
RESEARCH, PRACTICE, AND POLICY 



Introduction 



Although the findings from the second-year study resulted in 
an interesting set of conclusions regarding the process and content 
of learning-in-work environments, the question still remains, "In 
the light of what is known, how can the findings contribute to 
policy decision making and future research?" 

Even though this study is a preliminary investigation, 
several provocative findings were revealed; nevertheless, prudent 
researchers are corredt when they caution that the findings were 
derived from an imperf^iCt data set. This situation evolved from 
the fact thaiS^ students; that were not randomly assigned to the two 
learning environments and the dependent measures were standardized 
achievement test scores, which may not have been the most appropriate 
outcome measure either in terms of the content to be learned by 
the student or its psychometric properties. For example, selected 
test items maximize the discrimination among student populations 
at one given time, rather than measuring change in performance 
over a period of time; the small sample size reduced the povzer 
for detecting student learning patterns in the two environments; 
and the treatments were units of instruction occurring naturally 
in a large school district, thereby precluding a priori specifica- 
tion of cognitive variables that could have been used as covariates 
or independent variables for a better understanding of learning 
and retention of basic skills in alternative learning environments. 

To address the question of implications and recommendations 
for research, practice, and policy, commencs are incorporated 
from papers that were commissioned (Appendix B) to review the 
results of the first-year study. The commissioned researchers 
were David P. Ausubel, Distinguished Professor Emeritus, Ph.D. 
Program in Educational Psychology, Graduate School, City University 
of New York; Henry C. Ellis, Chairman, Department of Psychology, 
University of New Mexico; and Benton J. Underwood, Stanley G. 
Harris Professor of Social Science, Northwestern University. 
These men were commissioned to review the results of the first- 
year study and, through extensions of their research and learning 
theories, to provide a perspective for investigating the psycho- 
logical and pedagogical implications of learning and forgetting 
patterns in learning-in-work and traditional environments. 
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The investigators aclcnowledge that the interpretations of 
the papers are theirs and accept responsibility for any misin- 
terpretations that may be perceived by the coimnissioned authors. 

Txaplications for Research, Practice and Policy 

No longer is fomial schooling limited to learning in a 
classroom environment. Not only do youth have the opportunity 
to choose their learni vg environments ^ but educators concentrate 
on designing and impleiaenting alternative environments that seem 
likely to maximize student learning. Thus, a key question emerges: 
"For whom, -^en, and under ^at conditions are alternative learning 
envircnments effective in promoting the learning and retention of 
basic skills?" As a function of this research effort with its ^ 
focus on an investigation of the learning and retention of basic . 
skills in a learning- in-w^rk environment, four issues seem to be 
related to this question: the relationships between learning in 
a classroom environment and in a work environment, the relation- 
ships between environmental complexity and the learning environ- 
ment, the interaction of student characteristics and environmental 
complexity;, and the relationship between learning outcomes and 
their measurement. 

Issuer The Relationships between 
Environments and Learning 

For this investigation, the environments studied have two 
factors: the learning process itself and the physical environ- 
ment where learning occurs. Based on the terminology of David 
Ausubel, the program structures for these two factors would 
appear as follows: 



FIGURE 16 



The Relationship between Environment and 
the Process of Learning for Two Types of Programs 



Learning 
Environments 



process of Learning 



Experiential 



Didactic 



Worksite 



EBCE ( Learning- in-Work ) 
(80% time) 



Learning Center 



EBCE (20% time) 



Classroom 



Traditional (100% time) 
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Thus, in this study we have equated the learning- in-work 
environment with experiential learning and worksite exposure; 
and, the traditional learning environment with both didactic 
learning and classroom exposure, ,In reality, neither program 
is absolutely one or the other, and this fact highlights one 
difficulty in measuring the acquisition of basic skills in 
naturalistic settings. Extrapolating further from Ausubel's 
paper and including the dimensions of reception and discovery 
learning with meaningful and rote learning, we depict the 
relationship among the variables as follows: 



..FIGURE 17 

Reception-Discovery and Rote-Meaningful 
Dimensions of Learning 
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Definitions (Ausubel , 1980, p. 9): 

Reception Learning - the principal content of the material to be learned is 
presented to the learner in a nore or less final form and he/she need only 
-integrate it into his/her cognitive .structure for the purposes of retention 
and transfer to new learning experiences. 

Discovery Learning - the principal content of what is to be learned 'i.e., 
new successful problem- solving proposition) must first be discovered by 
transforming relevant background knowledge (previously acquired concepts, 
facts) in such a way as to constitute a means to the end specified in a 
problem-setting proposition. Once an acceptable problem-solving proposition 
is discovered, it is then internalized in precisely the same way as in recep- 
tion learning. 

Meaningful Learning - (1) if the student employs a meaningful learning set to 
his/her existing structure of knowledge and (2) if the learning task itself 
is potentially meaningful. 
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Although the addition of these variables to the environment 
in which learning occurs adds precision to the type£5 of questions 
that can be asked and researched, it increases the complexity of 
attributing changes in performance to single environments alone. 
However, in reference to our central question and this particular 
issue, the following types of questions can be raised: 

• Which students (e.g., cognitive style, age) 
learn through reception learning and which 
learn through discovery learning? 

• Do experiential-type programs increase student 
exposure to discovery learning? 

• Do worksite environments stress discovery 
learning more than reception learning? 

• What is the proportion of experiential to 
didactic learning in worksite environments? 

• Is there an interaction between basic-skill 
acquisition and retention through experiential/ 
didactic, reception/discovery learning and 
environment? 

• Meaningful learning is acquired by which students 
in which environments and under what conditions 
(experiential/didactic, reception/discovery) ? 

While our study only scratched the surface with regard to 
quescions of this type, we do have evidence, even with metood- 
ological reservations, that math skills learned in a learnmg-in- 
work environment may be learned and retained in ways that are 
different from those learned in a traditional environment 
(figure 3) . Furthermore, evidence from the CLPC of the math 
process variables (figure 15) suggests that the decrement 
observed at post-test for the learning-in-work students may be 
due t-o different causal cognitive processes that operate in the 
tv/o environments. With regard to reading comprehension, our 
evidence suggests that student performance in either environment 
is almost equivalent. 

Because students in the learning- in-work environment are 
bringing ten years of past school knowledge and learning 
processes to the workplace, the concept of transfer is an 
important issue. VJhile we can,not be certain, one possible 
explanation for the decrement in the math scores of these students 
at post-test may be attributable to interference from past school 
learning. Thus, for students who are making the transition to 
the workplace as well as for those who are experiencing other 
alternative environments, transfer variables not present m 
the classroom environment may be operating in such a way 
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as to interfere with math performance at post-testing. Two such 
variables suggested by Henry Ellis 1980, p* 9) are depth of 
processing and cognitive effort. For example, due to the nature 
of learning activities, students in the learning-in-work environ- 
ment may not process the information they have learned at the 
same depth or at the same intensity as students in the classroom. 
Thus, while increased p(*jrf ormance (higher math achievement) is 
important for classroom learning, learning at a workplace may 
require less depth, of processing or cognitive effort for student 
success, thereby resulting in lower math achievement at post- 
testing. If such mediating cognitive variables are operating in 
worksite environments but not in other learning environments, 
then they should be included in future research for studying 
basic-skill acquisition and retention * 

Issue: The Relationships between 
Environmental Complexity and Learning 

The second issue of environmental complexity and its rela- 
tionship to learning environments refers to 'those characteristics 
inherent (or perceived) in an environment that^ affect student 
performance. These characteristics are further delineations for 
classifying and studying learning environments. This issue was 
addressed because our prior knowledge of learning- in-work 
programs suggested that two characteristics, participation and 
negotiation, distinguished this environment from traditional 
learning environments. For example, learning tasks were nego- 
tiated by the student with the worksite mentor or program coor- 
dinator, affording students the opportunity to decide and help 
structure what they wanted to do. In the EBCE program this proces 
was formalized through the completion of the activity sheets. 
Since activities at the worksite vrere completed with other 
workers, students participated with adults in successfully 
completing tasks. 

Much of the discussion in this section results from conver- 
sations and correspondence between Harold Schroder and the 
project staff. Initial thinking regarding environmental charac- 
teristics that can affect performance centered on the contrast 
]3etween environments that "generate concepts" and those that 
"give concepts." In "concept generating" environ'Tients , according 
to the theory, students learn to search for information, generate 
ideas, try them out, and receive feedback. The hypothesis was 
that "concept generating" environments deve^Xqp skills that later 
may also help students acquire knowledge. In Henry Ellis's 
commissioned paper he called attention to the generation effect 
in recall (Slamecka and Graf, 1978) . The basic finding was that 
if subjects have to generate an answer rather than simply to 
remember a solution, recall is enhanced. As Ellis suggests, 
(1980, p. 7), these findings support the concept of learning- 



by-discovery and would appear to be operative in a learning-in- 
work environment. To some extent the generation effect appears 
to be similar to our notion of "concept generating" environments. 

The concept of environmental complexity may also be related 
to the means (process) and ends (goals) of the learning task as 
defined bv the parameters of the learning environments. The 
traditional learning environment is designed to maximize the 
acquisition of knowledge. In this environment the knowledge 
(content) goals are specified and taught explicity (means). In 
the learning-in-work environment the ends (goals of learning 
task) are negotiated between student and coordinator, thereby 
providing a structure for the task but learning some of the means 
(hov; activities are completed) unstructured and the student at 
the worksite rather than waiting for instructions, must initiate 
action to complete the tasks. The point is motivation of students 
should be preserved so that they may develop a capacity to make 
judgments and decisions necessary to act with personal independence, 

To furnish a measure of environmental complexity for the 
two environments in this study, the Learning Environment 
Questionnaire (LEQ) was developed. Analysis of this instrument 
suggests three factors related to student perception of the 
learning environment: (1) an opportunity for generating concepts 
or engaging in a variety of experiences, (2) a structure that per- 
mits learning and negotiating with adults, and (3) gratification 
(feeling of self-confidence) for initiating and carrying out 
work. Environmental complexity is suggested when higher levels 
of the factors are present (or perceived) in the environment. 

Data from this study suggest that the learning-in-work _ 
students perceived more environmental complexity (as determined 
by higher test scores) than students in the traditional _ environ- 
ment perceived. It also suggested that student perceptions of 
environmental complexity affect math performance at post- and 
follow-up testings. Overall, since higher levels of perceived 
environmental complexity resulted in slightly lower math per- 
formance, then if these environmental characteristics are 
considered important for learning, math achievement (as 
currently structured and measured) may be better in those environ- 
ments not emphasizing factors measured by the LEQ. 

Another environmental characteristic studied for this study 
was the exposure of learning-in-work students to worksites by 
the WTG classification. Although math differences were observed 
at post-test for the WTG's of Artistic and Social, these 
variables showed similar patterns of differences at pre-test. 
Thus, for this sample, there is doubt as to the efficacy of VTIG 
classification as a causal environmental variable. 
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The implications of this issue may best be stated by the 
question "In accordance with individual characteristics/ to what 
extent can alternative learning environments enhance performance 
over what can be attributed to personal characteristics alone?" 
Although basic skills (especially math) were the dependent variabl 
for this study, outcomes or performances other than math should 
be investigated. Thus, environmental factors such as participa- 
tion, variety, autonomy, and feedback may be differentially 
related to the ac:^ isition and retention • of subject-matter content 



Issue; Interactions between Persona l 
Characteristics and Environmental Complexity 

The issue of interactions between personal characteristics 
and environmental complexity is related to a growing body of 
literature directed at identifying the interactive nature of 
the personal and environmental determinants of behavior. Thus, 
is the interaction of personal characteristics vzith environmental 
complexity a more pov/erful predictor of performance than is 
either variable alone? 

For this study, the personal characteristic of cognitive 
style was measured by the Paragraph Completion Test (PCT) . The 
PCT yields an integrative complexity score for the individual's 
ability to generate ideas and to organize ideas or concepts in 
a hierarchical system. Results showed that higher integrative 
complexity scores were associated with higher math achievement 
at post- and follovz-up testings for students in both learning 
environments. However, the predicted interaction between 
cognitive style and perceived environmental complexity (LEO) 
on math performance was not observed for tl^^s sample. Even 
without the predicted interaction for this study, there is 
sufficient evidence to support research into the interactive 
nature of personal and environmental characteristics. Such a 
framework for studying the interactions effect is depicted in 
figure 18. ^ 



FIGURE 18 
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Implications are for research questions of the followin 
types : 

• Is performance maximized when the personal 
characteristics of cognitive complexity is 
matched to the same level of environmental 
complexity? 

• What dependent variables (performance) are 
best predicted by the interaction? 

• Do specific groups of students prefer one 
environmental complexity over another? 

• Are specific groups of students exposed to 
only one type of environmental complexity 
to the exclusion of others? 

• Hov7 does performance vary as a function of 
the individual's cognitive style? 

• Are experiential-type programs more environ- 
mentally complex than didactic-type programs? 



Issue: The Relationships of Learning 
Outcomes and Their Measurement 

The major issue is the appropriateness of the criteria for 
successful learning. For this study, increased achievement of 
math and reading comprehension skills was the learning criterion. 
As discussed earlier, this criterion is - appropriate for the 
traditional classroom learning environment. However, the mere 
lack of increased scores for the learning-in-work students may 
not imply that learning did not occur, but rather that the 
work environment, because of its complexity, caused decreased 
scores and produced increases in other outcomes not measured in 
this study. Thus, the criterion of increased cognitive knowledge 
may not be the appropriate learning ^criterion for a work environ- 
ment . 

Another factor to be considered is the interpretation of the 
findings within a classical framework of learning and retention. 
In this study, nine months served as the learning interval and 
three months as the retention interval. Most classical studies 
use shorter time frames for the learning and retention intervals 
with precise control over the learning task and random assign- 
ment of subjects. In effect, we are overlaying group data and 
macro time intervals on the micro cognitive process studies in 
the classical learning and retention paradigms. Thus, not only 
must we overcome the problem of separating the non- school effects 
from school in the presence of the two types of effects (especially 
in non-experimental designs) , but we must find ways to apply 
learning theories (classical or contemporary) to alternative 
learning environments to understand the tradeoffs of learning 
and retention of basic skills. 



Recommendations 

The preliminary research from this two-year effort has 
resulted in findings that support different patterns of learning 
and retention of basic skills as a result of experience in one 
of tv7o learning environments: traditional classroom or learning- 
in-work. At--this stage of the research' effort, we prefer to 
consider our research as a line of inquiry. With this view, we 
will offer recommendations for research, practice, and policy. 
Our findings to date, while interesting, do not justify a 
specific "do" or "don't" for researchers or practitioners. 
Rather, they suggest alternative ways to think about the learning 
and retention of basic skills. The*" recommendations assume that 
alternative learning environments are under investigation. 
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Research Reconunendations 



• Expand evaluation studies to include a retention 
interval or design the studies for longitudinal data 
collection, thus increasing the power of the research 
design to explain environmental effects on skills being 
learning. 

• Include other measures of outcomes in addition to basic 
skills to determine the effectiveness of alternative 
learning environments. 

• Use controlled experiments to test the hypothesis that 
there is an interference of past learning with learning 
occurring in the workplace. 

• Use environmental complexity characteristics not only 
as independent variables to describe the results of new 
curricula, but also as independent variables to predict 
which environmental factors are related to the learning 
of basic skills. 

• Test the viability of the transfer concepts of 
processing and cognitive effort for students in 
learning- in-work environments as possible explanations 
for decreased learning curves and increased retention 
curves. 

• * Design studies to test for the interaction between the 

personal characteristic of an individual's ability to 
search for information and use it effectively to solve 
problems and the perceived complexity of the 
environment. The hypothesis to be tested is that 
productivity (amount learned and retained) can be 
increased when there is a match between an individual's 
cognitive style and the level of environmental complex- 
ity. 

• Design methods to assess learning environments accord- 
ing to three or four independent dimensions. 

• Design criteria measures that are sensitive to the 
assessment of cognitive skills as well as to content 
acquisition. 



Fducational Practice Recommendations 

• Curriculum developers should not only design the 
curriculum but also design the environment (e.g., 
variety, autonomy/ feedback) for learning basic skills^. 



Educational Practice Re coinmenda t ion s (continued) 



• Alternative learning environments should be classified 
in such a way as to provide students with real choices 
in the selection of learning programs* 

# Students should be encouraged to sample or test 
alternative learning environments to determine under 
which conditions they function best* 



Policy Recommendations 

# Include in the regulations for federally-sponsored 
evaluations the use of environmental complexity and 
cognitive style measures so as to better understand the 
effects of environments on learning. 

# Fund research to develop/validate dependent measures 
appropriate for alternative learning environments. 
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LITERATURE REVIEW 



The literature reviewed for the present investigation centered 
on two categories that represent the different conceptual aspects 
of the phenomena under investigation. ' Reviewed first is a 
collection of laboratory-oriented research on basic learning and 
retention processes. This is followed by a review of studies on 
the relationship of personal and environmental characteristics 
to achievement, with emphasis on approaches that utilize the 
person-environment interaction model. 



Learn i ng/ Re t en t ion Re s e arch 

A wide variety of "studres.':^^ learning and 

retention in an education, context would seem to have relevance 
to the present investigation. Although there is an apparent 
lack of reseaafch dealing specifically with the retention of 
reading and ^mathematical competencies as a function of the 
particular characteristics of the learning environment under 
study, the principles derived from related research may be 
applicable to the present investigation of the learning-in-work 
and traditional environments. Ausubel (1968, p. 83-115) 
asserted that learning and retention of material is superior among 
those who have been given meaningf u 1 , as opposed to rote-memorized 
material. The basis for this finding centers on the following 
promises: (1) meaningful material is more easily relaired to 
information previously learned; (2) meaningful material can be 
incorporated vjith the learner's store of information more 
easi^ly 'than material simply memorized by rote and unrelated to 
other stored material; (3) retention of meaningful material is 
facilitated by the fact that there is more information 
available to retrieve, since meaningful material is learned 
better originally. The overall effect of meaningful 
information stored, is a more gradual retention decrement. 
Material which is "rote-learned" was observed to be subject to a much 
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steeper forgetting gradient as a consequence of poorer original 
learning. 



Gagne^ (1978) offered further insights on the nature of 
learning/retention for these two environments: EBCE or 
learning-in-work vs. traditional. The following model illn5=;trates 
the hypothesized conditions of learning/retention: 

INTERVENING 



LEARNEP 



jEARNIN( 



H I ST ORYl^ VINTE RVAI 




RETENTION 
TEST 



Under this conceptualization of the learning/retention process, 
pre-learning history refers to the events preceding the 
learning interval under study. The learning interval is the 
time given to transmitting the material to be learned. The 
retention period consists of the time between the learning 
interval and a follow-up retention test. Events which occur 
during this period are seen as able to control (through 
interference) the exact level of retention possible or the 
level appearing on the retention test. 

Research on the components of this model is seen to have 
implications for the present study. Improved retention was seen 
as 'a result of greater prior experience (learning history) 
in studies by Johnson (1973) . Scores on recognition of items were 
seen to suffer from interference created. by previously learned, 
.similar items (Sulin and' Dooling , 1974). However, in a 
'^laboratory study, Shuell and Keppel (1970) found no difference 
in later retention of learning from unequal prior 
learning. Overall, it appears that prior experience with the 
learning tasks should facilitate subsequent retention, . although 
this relationship can depend on the particular recall task at 
hand (i.e., recognition vs . application) . 

Research oh practice or usage of learning, which occurs during 
the retention interval, has revealed some clear trends. Boker 
(1974) and later Anderson and Biddle (1975) demonstrated a 
strong relationship between the nvimber of chances to apply the 
learned material and subsequ'^nt retention. The relationship 
indicated that in6reased practice leads to superior retention. 
The beneficial effect of using the acquired material in other 
non-school situations is borne out by later performance on 
a retention test. La Porte and Voss (1975) reinforced this- 
finding, demonstrating that the best results on retention 
tests are achieved when learners can use the information learned 
and can receive feedback on their performance. This fact was 
interpreted by La Porte and Voss as suggesting that the effect 
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resulted from strengthened retrieval paths and to a learning 
facilitated .by feedback. Further support for the beneficial 
effects of practice was offered by Rickards and Friedman (1978) , 
who found that forcing students to use previciisly l<)arning 
material leads to better retention performance. This effect was 
explained as a strengthening of the original memory, which results 
from the usage of the material. The effect of practice was further 
generalized by Kulhary and And erson (1972) f who found that even 
delayed practice facilitates subsequent retention. In the con- 
text of the present study, these findings may be inte:s?preted as 
predicting superior retention performance both at th(?> ^nd of 
the school year and at the end of sutnmer for those sttsdents who 
have the greatest opportunity to use previously learnf^^S material 
(e.g., the EBCE students, who apply concepts previously learned 
to the on-site experiences) . 

In a study of the most effective learning environment 
dimensions, Weber (1976) conclxaded that learning is maximi:?ed 
under specific conditions: (1) if the learning materials 
allov7 considerable feedback; (2) if teachers are highly directive 
and have the ability to provide feedback on' specific learning 
tasks; and (3) -if situations in teacher-student interactions are 
structured to provide immediate feedback. As applied to the 
present study, these results would suggest that learning will 
be facilitated for the group having the greatest access to 
teachers giving task-references and to specific feedback as 
soon as possible - 

Of the two learning environments surveyed here, the EBCE 
condition would seem more effective because of (1) access it 
provides to learning materials in a concrete form, thereby 
facilitating observation of performance; (2) access it provides 
for resource persons available for giving specific directive 
feedback on job performance; and (3) a one-to-one ratio of 
supervisor (resource person) to student for each site, thereby 
allowing the closest possible monitoring of and feedback on 
performance - 

Kerrigan (1976) compared the effectiveness of traditional 
and "individualized" learning systems in producing achievement 
in mathematics instruction. The individualized approach was 
characterized as structuring the progress and content of the 
learning material to fit the learning rate and style of each 
student. . Results of this study revealed that the individualized 
approach produces the highest level of overall math achievement 
at the end of the course and also achieves the greatest 
improvement in scores for low-ability students in the program. 
To the extent that the EBCE envirom^^^ent offers individual pro- 
grams for learning material, it may be expected that students 
(especially lower-ability students) enrolled in these 
programs should exhibit the gre.^-test gains in learning 
mathematics competencies. 
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Person/Environment Research 



A wide variety d examined the joint 
contributions of personal characteristics and environmental 
structure as they affect performance. These studies may be 
further grouped as follows: studies dealing with the 
relationship of "person" variables (i.e., cognitive style) 
to task performance and studies dealing with the interaction of 
the environment and the person characteristics. It is hoped 
that such classifications of persons and environments will 
facilitate understanding of the processes operating in the 
present study. 

Cognitive-Style Studies 

"Gray and Knief (1975) investigated the relationship of 
cognitive style, defined as stable preferences of individuals 
with respect to conceptual categorization and perceptual _ 
organization of the external environment, to academic achieve- 
ment. Results indicated that the strongest effects were 
observed when the correlations of cognitive style to achievement 
were computed on an individual classroom basis. However, when 
overall results were completed, averaging across the classes, 
these effects V7ere trivial. No clear- pattern was evident for 
the classroom-level analyses. It was concluded that an mter- 
^active er^^^^ (changing relations as a function of classroom 
environment) was present, which would account for the lack ot 
clear "main effects" of cognitive style on performance. 

Davey (1976) investigated the relationship of cognitive 
style to academic achievement, concluding that the match of the 
students' cognitive styles to the instructional method was a 

, critical factor in maximizing performance. Learning by 
discovery and other "individualized" instructional programs 
were seen to be better suited for more complex, field-independent 

, students. More structured programs were seen to be better for 
less complex, field-dependent students. 

- Russell and Sandilands (1973) investigated the relationship 
of conceptual complexity (Harvey, Hunt and Schroder, 1961) or 
cognitive style to a variety of biographic and demographic 
variables. Results indicated that both the paragraph 
completion test (Schroder, Driver, and Streufert, 1967) and the 
Interpersonal Topical Inventory (Tuckman, 1966), which were, 
used as measures of cognitive style, were unrelated to age, 
sex, birth order, and grade-point average. 
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Interaction Studies 



A growing body of research has emphasized ..the applicabili 

of a paradigm which includes both individual differences and 
environmental effects in a joint capacity as determinants of 
behavior (e .g . , Cronbach^ 1967). Hunt (1975) provided a 
strong statement of the necessity of including both types of 
variables^ particularly when the effects of different learning 
environments are being investigated. Conceding that research would 
not be likely to produce findings that are applicable to 
"persons in general" and that many personal characteristics would 
be expected to have differential effectiveness in different 
environments^ Hunt argued for future research directed at 
identifying the interactive nature of the personal and 
environmental characteristics of behavior. 



With respect to the variables of interest in the present 
study (i.e./ environmental and conceptual complexity), several 
studies have suggested that an interactive effect Hiay be 
present. For example, Streufert, Suedfeld, and Driver (1965) 
using a simulation task, found that an interactive effect 
was present: the relationship of environmental complexity to 
performance on the task was different as a function of the 
conceptual complexity of the subject, since more complex 
subjects were less influenced by the environment than less 
complex subjects. Staszkiewic^ (1977) examined the relationship 
of cognitive complexity and "classroom climate" (a measure of 
the degree of direction given by the instructors) to 
mathematics achievements in a sample of high school students - 
Results indicated that an interactive effect existed between 
these variables, since more cognitively complex students 
scored higher than less complex students in situations 
characterized by less teacher direction. 

A review of the literature relevant to environmental c.nd 
individual complexity (Goldstein and Blackman, 1977) indicated 
that an interactive relationship may be expected between 
these variables. The predictions of superior performance by 
more complex individuals at all levels of environmental 
complexity, combined with optimal performance for complex 
subjects occurring in more complex environments than for 
less complex subjects, have been supported in a variety of 
simulation studies (e. g. , Karlins, Coffman, Lamm, and 
Schroder, 1967; Streufert and Schroder, 1965). 
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Summary 

While no research on learning and retention was discovered 
that directly examined the conditions under study in the 
present investigation, the references above do appear to provide 
a framework on vzhich predictions of learning and of retention 
performance of EBCE as compared with traditional students may 
be based. 

In terms of the Gagne^ . . . model, generalization to the 
present investigation would identify (1) .learning history as 
all prior school performance, to be examined for inequality at 
pre-test (on CTBS scales) ; (2) learning interval as the period 
from grade 11.0 to 11.9, during which students will be exposed 
to the two learning environments with a "learning test" 
administered at grade 11.9 to determine the degree of equal or 
unequal learning; (3) retention test as the - test given at grade 
12.0 to determine the degree to which students have retained 
the information learned in the junior year and the degree to 
which they have assimilated this information with previous 
learning; and (4) intervening events as those facilitating 
factors leading to assimilation of material and the inter- 
fering factors leading to poor^ transfer or assimilation of 
material - 

Relevant research would, suggest that improved- performance 
on the learning and retention tests results from the frequency, 
concreteness , and meaningf ulness of feedback, the chances for 
practice' (use) of learned material; and the c^ontingent feedback 
during both the learning interval and the retention interval. 

Research has emphasized the need to investigate the 
interactions of personal characteristics (i . e ., individual 
differences) with characteristics of environments.- An 
application of this methodology relevant to the present 
investigation has toen the research investigating the interaction 
of environmental complexity and individual complexity. 
Cons idera^ble support is pre.sent, at least on controlled settings, 
for the notion that the relationship of personal complexity to 
performance will vary as a function of the complexity of 
environment in vzhich the behavior occurs. 
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Description of the Learning-in-Work Program 

In order to provide the reader a better understanding of 
the learning-in-work program under investigation, a description 
of its major structural features is provided. The program is 
the Appalachia Education- Laboratory (AEL) model of 
Experience-Based Career Education (EBCE) located in the Anoka- 
Hennepin District II, Anoka, Minnesota. The Anoka-Hennepin 
District II public schdols were selected as the Minnesota (EBCE) 
demonstration site. The program was implemented during the 
1977-78 schooll-year, and students from the three high schools 
in the district began participation during the second semester. 
An anthropological description of the program can be found in 
Learning and Work Programs: Transitional Educative Cultures ^ 
(Crowe and Twarog, 1979) . , 



Background Information 

The EBCE program is designed as an academically oriented 
community-bas'"ed career exploration program bridging the gaps ^ 
between school and the community and between study and 
experience by emphasis on basic skills and "varied career 
explorations. . 

The program at Anoka consists of three learning 
coordinators. (LC) , a site analyst, and a program director. The 
learning coordinators are primarily concerned with supervising 
the individual learning activities of students. The 
major vehicle for designing student experiences is the activity 
sheet negotiated jointly by the student! and LC; and during the 
course of a year, a student completes approximately one hundred 
of these sheets. The site analyst is primarily responsible 
for maintaining and upgrading the work placements and for 
assisting students in choosing a site for career exploration. 
During a school year, students may have from three to twelve 
site experiences depending on their preferences and purposes for 
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selecting the sites. Each site (workplace) provides a 
resource person (RP) responsible for supervising and guiding 
the learning of the student at the placement. 

Students spend the first week becoming oriented to the 
program and assessing their career interests and academic 
needs.- After orientation, their week is divided with fou7r days 
spent at ; the site and one at the learning center. Time at 
the site parallels normal school hoilrs, from 8:30 to 2:30. 
Learners do not receive pay for the worksite experiences but 
do earn academic credit toward high school graduation. 

Description of the Instructional Process 

What is the instructional ' sequence of students? Figure 1, 
illustrating the macro-flow of student instructional 
activities, was developed from an analysis of the Student 
Program Guide (1976) and our observations of the program. Our 
view is that major steps are six-fold: (1) assessing career 
interests and aptitudes; (2) assessing academic interests and 
needs; (3) integrating career interests and academic needs; 

(4) designing learning experiences; (5) implementing the pl^-n; 

and (6) evaluating the experiences. 

Before each step is described, it is important to document 
one of the assumptions about what students are learning in 
the program. The assumption was that students were learning 
traditional subject-matter content through experiences at the 
worksite. In this EBCE model , traditional school subject matter 
is demonstrably related to both the Worker Trait Groups of the 
Dictionary of Occupational Titles (DOT) and the twentv-eight 
EBCE courses. An example of the chart that students use to 
relate school subject matter to occupations through worker 
trait groups is as follows: 

Mat:hematics (Student Program Guide, 1976, p. 10) 

School] Subjects Selected Wo rker Trait Groups 

General Mathematics 9 , 13 , 16 , 24 , 44 

'1 . . . ■ 

Algebra, Geometry 13, 44, 55, 59, 60 

Trigonometry 13, 44, 58, 60, 88 

Student course selections in EBCE are also related to 
school subject matter as reflected in the following three 
selected EBCE descriptions of courses appearing in the Student 
Program Guide (1976, p. 83-89): 

Student Program Guide , Charleston, W.Va. : Applachia Educational 
Laboratory, 1976. 
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"step 6 

EVALUATING EXPERTRNr 




Evaluation of 
Learning Activities 


Individual Program Management 
Form 



STEP 5 

IMPLEMENTING PLAN 



Spends 4 days at worksite 
Spends 1 day at Learning C enter 



STEP 4 
DESIGNING 
LEARNING 
EXPERIENCES 



Translating general choices 
into specific objectives, 
activities, products, time 
tables, points for academic 
credit 



ACTIVITY SHEET 



indicates instructional material available for 

student to make linkages 
indicates sequence 



ALGEBRA is a standard mathematics course with which you are 
probably familiar. It is strongly related to some 14 Math 
Subrrconcepts (Logarithms & Exponents, Functions, Equalities & 
Inequalities , etc. ) in the EBCE system. It also relates to 
some 6 Natural Science Sub^concepts (Atomic Structure, 
Electricity, etc.), and to 8 English/Communications Sub-concepts 
(mostly dealing with Research, Reading Comprehension and Reporting 
Skills) . Some application to community sites is possible 
(e.g., a chemical research laboratory), but you should expect 
much skills-practice and problem-solving through extensive 
use of in-house materials. Many cf the relationships in EBCE 
of Algebra to other course structures involve skill-building 
for more sophisticated mathematical and scientific applications. 

FUNDAMENTAL MATHEMATICS focuses on "brushing up" on such 
basic math skills as computation (adding, subtracting, 
multiplying, dividing), percentages, conversions, measurements, 
and ratio and proportion; these are skills that you will 
need no matter what career you choose. This course relates 
s-trongly to all five curricular areas: 11 Math Sub-concepts 

(Whole Niombers, Ratio, Tables & Graphs, Consumer Math); 8 
Natural Science Sub-concepts (matter & Energy, Electricity, Complex. 
Machines); 12 English Communications Sub-concepts (Reading, 
Writing,, Research, Reporting); 3 Social Science Sub-concepts 

(Price'f Physical & Human Geography) ; and 2 Career Education 
Sub- -loncepts (Personal Aspects of Work and The Work Place • You 
can pick up and practice these fundamental math skills at 
virtually every experience s:;.te, ranging from stores to 
newspapers to governmental agencies. 

, MODERN TECHNOLOGY is perhaps EBCE ' s broadest course 
area, for it is concerned with the impact of technology on all 
aspects of our society. It dan involve the influence of ne^'^ or 
better ways of performing technical tasks (e.g., machinery, 
equipment, science, computers); our political, economic, and- 
social institutions (e.g., elections, business cycles, labor 
unions, ecology); our , communications processes (TV, radio, 
magazines) ; our career patterns and trends (obsolescence, new 
care^XF>) r or our individual and interpersonal relations 

(mental healt'h, leisure time, travel) . This course area relates 
heavily to all five curricula: to 17 Social Science 
Sub-concepts (Distribution , Decision-Making , Geography, 
Philosophy; to ■ 12 Natural Science Sub-concepts (Atomic 
Structure, Light , Biochemistry, Engineering Systems); to 
12 Math Sub-concepts (Logarithms , Whole Numbers, Probability/ 
Statistics) ; to 12 English/Communications Sub-concepts 

(Research, Writing, Reading, Speaking, Reporting) ; and to 

2 Career Education Sub-concepts (The Work Place, Job Mobility/ 

Security) * You can study some aspect of Modern Technology on 
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virtually any experience site after careful selection of the 
aspects most appropriate for you. 

The purpose of including this specific information is to 
show that the program explicitly relates traditional subject- 
matter disciplines to occupations and worker traits required for 
a job, to experiences at the worksite , and to the courses that 
students choose. Thus, we have some assurance that students 
have an available opportunity to learn and retain traditional 
mathematical and readi^rg concepts (skills) . 

Step Ir Assessing career interests and aptitudes ^ During 
this phase students may choose to complete several diagnostic 
instruments designed to assess their career interests and 
aptitudes. Upon ccmpleting these assessments, students then 
review "Experience Site Learning Guides" for the purpose of 
determining a match between their career interests and the 
learning activities at a worksite* The Guide details specific 
information about the worksite career opportunities and job 
duties (keyed to the D.O.L. worker trait group). 

Step 2: Assessing academic interests and needs .. During 
this phase, students decide which EBCE courses they will select. 
After choosing a course, they refer to the EBCK course and 
interest area matrix for the purpose of choosing the academic 
sub-concepts and interest areas that will define the parameters 
for developing short-term learning activities. 

Step 3: Integrating career interests and academic need's . 
Althougn tJiis ±s not a formal step in the instructional process , to 
the researchers it represents a time when students can reflect 
on their decisions and thinking about the kinds of activities they " 
would like to have - at the worksite. 

Step 4; Designing learning experiences . The major tool 
for this stage is the activity sheet, that students and 
learning coordinators use to translate the student choices made 
in Steps 1 and 2 to specific learning activities:.^ The activity 
sheets spell out student tasks, estimate the time frame the 
activity will require, and furnish the basis for measuiring 
progress; They also relate subject-matter concepts to 
learning objectives and site experiences. Students may be 
completing up to five or six activity sheets at any given time^ 
Activity sheet assignments are expected to be- completed! at the 
worksite. That is, there are no homework expectations for 
students* 

Step 5; Implementing plan . After the activity^ sheet has 
been completed and accepted by the student and learning 
coordinator, it becomes a guide for student learning both at 
the worksite and the learning center. 
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step 6; Evaluating / The final step in the instructional 
process is the evaluation of the learning activities. That 
is conducted jointly by the learning coordinator and the student. 
Students are given the~ opportunity to do additional work if they 
desire a better evaluation. Upon satisfactory completion of the 
activity sheets students record their progress on an Individual 
Program ManageKient Form and begin a re-assessment of their career 
interests and academic needs. 
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TREATMENT (LEARNING- IN-WORK) AND 
CONTROL (TRADITIONAL LEARNING) STUDENTS 
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Uncorrected CLPC Results of Math CTBS-Bloom Scales 
for Treatment Students' Pre- and Post-Test Scores 



PRE-TEST 



POST-TEST 




KEY: Process Variables 

R = Recognition 

T = Translation 

I = Interpretation 

A = Analysis 



0.99 
1.12 
0.91 
1.02 



n = 38 
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Corrected CLPC Results of Math CTBS-Bloom Scales 
for Treatment Students' Pre- and Post-Test Scores 




KEY: Process Variables j^^ _ o.99 



* Z = -1.85, p = .06 
** Z = 1.80, P = .07 
*** Z = 1.31, p = .19 



n = 38 
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T = Translation = 0.91 

I = Interpretation i 

A = Analysis = 1-02 
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Uncorrected CLPC Riesults of Math CTBS-Bloom Scales 
for Control Students V Pre- and Post-Test Scores 



'PRE-TEST POST-TEST 




KEY: Process Variables K *^ = 1 60 

R = Recognition = 1.04 

T = Translation 2 ^ q ^5 

I = Interpretation 1 

A = Analysis = 0.90 



n = 42 



Corrected CLPC Resulti of Math CTBS-Bloom Scales 
for Control Students' Pre- and Post-Test Scores 



PRE-TEST 



POST-TEST 




KEY: Process Variables 

R = Recognition 
T = Translation 
I ^ Interpretation 
A = Analysis 



1.60 
1.04 
0.95 
0.90 



A Z = 2.52, p .02 



n = 42 
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Uncorrected CLPC Results of Math CTBS-Bloom Scales 
for Treatment Students' Pre- and Follow-up Scores 



PRE-TEST 



FOLLOW-UP 




KEY: Process Variables 

R = Recognition 

T = Translation 

I = Interpretation 

A = Analysis 



0.70 
1.44 
0.88 
1.06 



n = 30 



105 



ERIC 



ii 



6 



Corrected CLPC Results of Math CTBS-Bloom Scales 
for Treatment Students' Pre- and Follow-up Scores 



PRE-TEST 



FOLLOW-UP 




KEY: Process Variables 

R = Recognition 

T ^ Translation 

I = Interpretation 

A = Analysis 





= 0.70 


K 


= 1.44 




= 0.88 


< 


= 1.06 



* Z = -1.96, p .05 



n = 30 
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Uncorrected CLPC Results of Math CTBS-Bloom Scales 



for Control Students' Pre- and Follow-up Scores 
PRE-TEST — FOLLOW-UP 




KEY: Process Variables „ 1.44 

R= Recognition = 0.80 

T = Translation - 1 15 

I = Interpretation . i " * 

A= Analysis = 1.40 

A 



n = 28 
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Corrected CLPC Results of Math CTBS-Bloom Scales 
for Control Students' Pre- and Follow-up Scores 



PRE-TEST FOLLOW-UP 




KEY: Process Variables = 1.44 

R= Recognition = 0.80 

T= Translation = 1.15 

I = Interpretation ' 

A = Analysis = 1.40 



n = 28 
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Uncorrected CLPC Results of Math CTBS-Bloom Scales 
for Treatment Students' Post- and Follow-up Scores 



POST-TEST FOLLOW-UP 




KEY: Process Variables 1^2 _ q 

H. 

R= Recognition = 1.44 

T = Translation _ g 91 

I = Interpretation i 
A = Analysis = 1.09 

A 



n = 38 
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Corrected CLPC Results of Math CTBS-Bloom Scales 
for Treatment Students' Post-. and Follow-up Scores 



POST-TEST 



FOLLOW-UP 




KEY: Process Variables 

R = Recognition 

T = Translation 

I = Interpretation 

A = Analysis 





0.77 




1.44 








0.91 




1.09 



* Z = -1.35, p = .18 
n = 38 
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Uncorrected CLPC Results of Math CTBS-B loom Scales 
for Control Students' Post- and Follow-up Scores 



POST-TEST FOLLOW-UP 




T = Translation = 1.19 

I = Interpretation » 

A = Analysis = 1.18 

A 

n = 28 
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Corrected CLPC Results of Math CTBS-Bloom Scales 
for Control Students' Post- and Follow-up Scores 




KEY: Process Variables 




0.87 


R =»= Recognition 




1.26 


T * Translation 




1.19 


I " Interpretation 




1.18 


A = Analysis 


K' 



* Z = 1.34, p = .18 



n = 28 
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CROSS-LAGGED PANEL CORRELATIONS OF 
MATH CTBS FACTOR-BASED SCALES FOR 
TREATMENT (LEARNING- IN- WORK) AND 
CONTROL (TRADITIONAL LEARNING) STUDENTS 
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CLPC Results of Math CTBS Factor-based Scales for 
Treatment Students' Pre- and Post-Test Scores 




CLPC Results of Math CTBS Factor-based Scales 
for Control Students' Pre- and Post-Test Scores 
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CLPC Results of Math CTBS Factor-based Scales 

for Treatment Students' Post- and Follow-up Scores 



POST -TE ST FOLLOW-UP 
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CLPC Results of Math CTBS Factbr-b-ased Scales 
for Control Students' Post- and Follow-up Scores 
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CLPC Results of Math CTBS Factor-based Scales 
for Treatment Students' Pre- and Follow-up Scores 



RAW PRE-TEST FOLLOW-UP 




CLPC Results of Math CTBS-based Scales for 
Control Students' Pre- and Follow-up Scores 
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Introduction: Historical Context of the Learning -in-Work 
Approach to the Acquisition and Retention of Knowledge 



By way of introduction, it might be useful, in assessing, in 
attempting to explain, and in suggesting new perspectives for 
validating and extending the preliminary research findings of the 
Learning-in-Work Research Program of The National Center for 
Research in Vocational Education, to identify briefly'^ the more, 
significant trends in pedagogic theory and practice to which this 
research program is historically related. This is more than a 
mere academic exercise. For by placing this research in its 
historical context we can also hopefully identify the crucial 
variables involved in such learning, focus on, and take into 
consideration relevant available explanatory hypotheses, as well 
as generate psychologically tenable new hypotheses, and suggest 
new aspects of research strategy that profits from both the 
strengths and weaknesses of the research methodologies associated 
historically with these related psychoeduca tional movements. 

First, it is important to recognize that experiential 
learning is indisputably based on the first pedagog ic- strategy 
evolved in the course of cultural evolution. Further, it is 
still very much alive today as a significant coirponent of formal, 
vocational, and professional education. In such professions as 
medicine, dentistry, law, accounting, nursing, teaching, social 
work, engineering, etc., the apprenticeship system of instruction 
not only originated first but also co-existed later with formal 
professional schools until relatively recently as an alternative 
means of entry into the profession* The importance of practical^ 
naturalistic experience in formal, traditional schooling has also 
been recognized more recently in both college and high school 
work-study programs,: in career-oriented education, and in. 
ever-expanding internship programs in government, industry, and 
in various social agencies and institutions. A still more recent 
development has been th^ training of parapr of ess ionals in such 
areas as medicine, dentistry, nursing, and teaching that places 
relatively greater emphasis on the experiential as opposed to the 
formal theoretical aspects of training. 

In all of these pedagogic developments involving the 
acquisition of manual, social, and intellectual work skills, 
perhaps the chief pedagogic problem has been the relationship 
between the traditional theoretical classroom component and the 
practical , exper iential or natural istic component in terms of 
relative emphasis, timing, prox ima teness of relevance, 
correlation, and transfer. 

Closely related to vocational and professional education 
have been such educational movements as the "social utility" and 
"life adjustment." Apart from the philosophical dictum that the 
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only knowledge worhh acquiring is that which enhances adaptation 
to practical problems of living, the pedagog ic ;ra tionale has been 
an emphasis on the presumed motivational advantages of clear and 
perceptible relevance to current problems of personal and social 
adjustment. The theme of relevance, of course, more recently 
became the battle cry of the student revolt in the late Sixties 
and early Seventies. 

A third significant educational trend in experiential 
learning was the growth of the Progressive Education movement 
with its emphasis on "learning by doing" in "real-life" settings, 
on direct, concrete experience, on incidental or non-deliberate 
learning, on student par tic ipa tion in educational planning and 
evaluation, on the existing interests and endogenous motivation 
of pupils, on autonomous discovery and problem-solving, on 
student-oentered teaching, and on the avoidance of contrived or 
structured practice and drill. Many of these ideas were • 
borrowed from or were elaborations on the original 
predeter ministic notions of child development advanced by such 
educational philosophers as Rousseau, Froebel, and Pestalozzi. 
The original project and activity approaches of Progressive 
Education were the forerunners of such later movements as 
learning by discovery; "process," inquiry, and "heuristics of 
discovery" approaches to science education; and "teaching for 
creativity and independent thinking." 

The principal pedagogic difficulties generated by these 
various movements stemming from Progressive Education were the 
unwarranted identification of meaningful learning with 
self-discovery and direct, concrete experience; the confounding 
of the rote-meaningful and reception-discovery dimensions of 
learning; the unwarranted and gratuitous stigmatizing of 
expository instruction as both necessarily leading to rote 
learning and as authoritarian; the unrealistic denigration of 
structured and contrived learning experience, practice, and 
non-endogenous interests and motivations; and the neglect of the 
facilitating and integrative influence of abstract verbal 
concepts and principles in the acquisition and retention of 
knowledge and in problem-solving. 

The Fifties and early Sixties of this century were the 
heyday of the curriculum reform movements, particularly in 
mathematics and the natural sciences. These curriculum movements 
emphasized, for the first time in thirty years, that a given 
discipline consists of a hierarchically organized body of 
knowledge; that certain key explanatory ideas have organizing and 
integra tive proper ties in effectively transmitting this body of 
knowledge to pup ils ; and tha t the subject-ma tter content taught 
to pupils must reflect the most valid, up-to-date, and cogent 
thought prevailing in the discipline (Bruner, 1960; Finlay, 1959, 
1960 ) . 
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The curriculum reform movements foundered, however, because 
the relationship between didactic and experiential instruction 
was still not adequately understood (undue emphasis was still 
placed on the role of autonomous problem-solving or discovery and 
of direct, concrete experience in meaningful learning and 
retention); because they were not based on pedagogic principles 
derived from a comprehensive and self-consistent theory of 
meaningful learning and retention; because their subject matter 
materials were overly difficult and complex, due to insufficient 
pretesting for learnability and inattention to the presence of 
necessary background knowledge in pupils' existing cognitive 
structures; and because the effectiveness of the programs were 
inadequately evaluated and validated (for reasons to be discussed 
below) • 

Concomitantly with the rise and fall of the curricular 
reform movements, but completely ignored by them, a cognitive 
theory of meaningful learning and retention (Ausubel, 1958, 1960, 
1963, 196 8; Ausubel , Robbins and Blake, 1957) was being evolved 
which specified the nature and conditions of meaningful learning 
and rete^ntion as processes distinct from their rote counterparts; 
clearly differentiated between the orthogonal rote-mea^ningful and 
reception-discovery dimensions of learning; specified the 
cognitive structure and other cognitive (i.e., practice, 
instructional material, level of intellectual development, and 
intellectual ability, cognitive style), social, motivational, and 
affective variables influencing meaningful learning and 
retention; developed general pedagogic principles (e.g., 
progressive differentiation, integrative reconciliation) 
consistent with this theory and pedagogic devices (e.g., advance 
organizers) consistent with the principles; specified the 
relationship between the respect ive didactic and experiential 
aspects and roles of learning for different grade levels, levels 
of subject matter sophistication, and levels of professional or 
vocational objectives; and differentiated between the major 
categories of meaningful learning (representational, concept^ 
prepositional ) • 

Lastly, experiential learning was significantly influenced 
by the individualized instruction movement which recognized the 
importance of taking into account individual differences in 
general, as well as more specific and immediately prior, 
cognitive structure , intellectual ab il ity , cognitive style , 
motivational , per sonal ity, affective , and social var iables in the 
programming, sequencing, and pacing of learning experience. The 
possibility of handling the complex logistics of individualized 
learning experience for these multiple variables first became 
realistically realizable with the development of sophisticated 
and economically feasible computer-assisted instructional 
technology and with the growing acceptance of the evaluation and 
measurement concepts of "mastery learning" and criterion- 
referenced liieasures. 
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Theoretical Aspects of the Relationships between 



Didactic and Exper iential Learning 



In discussing relationships between didactic and 
experiential learning/ it is important ,at the outset to 
distinguish between two kinds of knowledge that can be learned 
both didactically and exper ientially r namely ^ (1) "applied" 
knowledge that has more . or less immediate practical relevance and 
applicability to the world of work (i.e. r vocational and 
. prof essional knowledge) and (2) "pure" knowledge which is 
acquired as an end in itself just for the sake of knowing or 
better understanding per sonally significant physical, biological^ 
behavioral, and social phenomena in one * s environment . 

Naturally these two kinds of knowledge are not dichotomous. 
Much "pure" knowledge, for exa^mple, ultimately has relevance for 
and applicability to practical mechanical, biological, 
educational, and social problems. However, the original 
motivation and intent for its discovery or learning are not 
necessar ily utilitarian; and its applicability to or relevance 
for such problems is more remote, thus requiring the 
consideration of additional variables relative to . the particular 
problem situation and objective in question; the generation of 
new theor^etical principles at an applied level of generality and 
relevance; and the performance of additional research in the 
appl_ied setting or framework. I am making this distinction 
explicit in order to emphasize that in this paper I am dealing 
only with applied knowledge intended to be learned and used in a 
vocational or professional context. - 



Reception-Discovery and Rote-Meaningful Dimensions of Learning 

It is also necessary to point out explicitly that just as 
pure, as well as applied, knowledge can be acquired in 
experiential settings, such settings include both discovery and 
reception learning, each of which, in turn, may. be rote or 
meaningful depending on the conditions under which the learning 
occurs. Both types of learning (i.e., reception a ndjdis^^ 
may be said to be meaningful (1) if the student enploys a 
meaningful learning set (a disposition to relate new learning 
material nonarbitrar ily and substantively ( nonverba timly ) to his 
existing structure of knowledge) and (2) if the learning task 
itself is potentially meaningful [(a) if it itself can be • 
nonarb i trar ily (i.e., plausibly or sensibly) and substantively 
(i.e., nonverba timly ) related to any appropriate cognitive 
str uctur e ( i .e . , manifests ^'logical meaning") and (b)if the 
particular learner 's existing cognitive structure contains 
relevant anchor ing ideas to which the new learning material can 
be so related.] 
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Thus, the acquisition of new meanings is an idiosyncratic 
product of the interaction between new logically meaningful 
material (e.g», subject matter) and relevant existing ideas in a 
particular learner's cognitive structure when the latter 
individual manifests a meaningful learning set. In reception 
learning the principal content of the material to be learned is 
presented to the learner in more or less final form and he needs 
only to integrate it into his cognitive structure for purposes of 
retention and transfer to new learning experience » In meaningful 
discovery learning on the other hand,, the principal content of 
what is to be learned (i^e., a new successful problem-solving 
proposition) must first be discovered by transformiing (applying, 
restructuring, reorganizing, reintegrating ^ analyzings- 
synthesizing) relevant background knowledge (previously acquired 
concepts, facts, and principles) in such a way as to constitute a 
means to the end specified in a problem— setting prop o-s it ion- The 
disposition to discover an insightful means— end; relationship 
constitutes the meaningful learning set in mjeaningful discovery 
learning or problem— solving . Once an acceptable problem— solving 
proposition is discovered, it is then internalized in precisely 
the same way as in reception learning • 

Just as reception learning was stigmatized in the past as 
rote, by definition, discovery learning was similarly held tO' be 
meaningful by definition. Thus, for at least thirty years 
(approximately 1930-1960), mathematics and science at the high 
school, college, graduate, and professional school levels were 
taught almost exclusively by paper-and-pencil and laboratory 
problem-solving exercises, inasmuch as expository teaching andi 
explication were ax ioma tically regarded as necessarily leading to 
rote learning outcomes, and problem-solving approaches were 
similarly regarded as necessarily leading to meaningful, learniing . 
The disastrous upshot of these pedagogic attitudes was; that an 
entire generation of students "learned" science and mathematics 
by memorizing solutions to "type problems," by substi tuting^ 
specific numerical values for unknowns in memorized formulas and 
equations, and by performing "cookbook" laboratory exercises and 
.manipulations of symbols without understanding what they were 
doing or why, and, hence, without acquiring any of the basic 
concept or prepositional meanings in these disciplines (despite 
solving successfully an acceptable percentage of the test 
problems involving these concepts and principles). 

Thus, although a certain minimal degree of pr oblemr-soLving 
experience is obviously necessary for learning the meth:<D)dblogy 
for a given discipline (i.e., for understanding how knowledge is 
discovered in that discipline) and also for learning how to* deal 
with, and overcome, difficulties in the appl lea t ion of _ 
didactically acquired concepts and principles to practical 
vocational or professional problems, it obviously accomplishes; 
neither objective unless the conditions of meaningful dlscTErvery 
learning are satisfied.— 
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Fur ther /v problem-solving experience is much too 
time-consuming to be used as a pr imary means of learning the 
basic subject-ma tter content of any discipline^ pure or applied. 
If students had to rediscover by themselves the major portion of 
the substantive content of a given discipline ^ they would 
obviously not have sufficient time to acquire more than just the 
rudimentary aspects of that discipline and^ thus^ would be 
incapable of solving even the relatively simple and routine 
practical vocational or prof ess ional problems involving the 
application of its theoretical content. 

Two additional disadvantages of the discovery approach to 
acquiring the substantive concepts and principles of a given 
discipline^ even under guided or contrived discovery conditions, 
are (1). that it requires greater intellectual ability to 
rediscover these concepts and principles than to understand them 
in a reception learning context and (2) that, in any case, even 
those students who are capable of such rediscovery require much 
more time to do so than would be the case if this substantive 
content were presented to them. It is evident, therefore, that 
inasmuch as abstract and general knowledge can be applied only if 
and after it is discovered, fewer students will have such 
knowledge available to a and only after varying periods of 
discovery time have elapsed, in comparison to their fellows who 
are exposed to effective expository instruction. The latter 
students, therefore, can acquire a much larger total body of 
abstract knowledge relevant for appl ica tion to problem- solving 
situations in an equivalent period of instructional time. A much 
larger proportion of these students are also successful in 
acquiring such knowledge; and this knowledge, additionally, is 
available for application much sooner. 



Relative Transferability of Substantive and Procedural Knowledge 

Even if discovery enthus iasts concede the gr ea ter ef f icacy 
of reception vis-a-vis discovery learning for acquiring the 
abstract, substantive content of a discipline, they are prone to 
insist that procedural knowledge, or knowledge of how to solve 
problems or how to discover new knowledge, gained through 
problem- solving experience, is more significant for solving the 
practical, vocational, or professional problems in an applied 
discipline than is relevant^ substantive knowledge. This point 
of view has spawned such approaches to science education as 
"process," "inquiry," "heuristics of discovery," and "junior; 
scientist" models. 

The rapid rate of obsolescence in science is also often 
offered as a rationale for these latter approaches. Since the 
content of what is taught today wiirbe obsolescent in fifteen 
years, the argument runs, students should be taught the 
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process rather than the content of science. Actually^ the rate 
of obsolescence in science is grossly exaggerated. Although the 
specifics of science, do change rapidly, basic principles tend to 
manifest impressive longevity; and it is more the case that their 
validity is not negated or superseded by, but rather subsumed 
under, new and more inclusive or sophisticated conceptual- 
izations. 

This argument is also strikingly reminiscent of the 
objection small boys frequently offer to washing their faces 
daily, namely, that they will only get dirty the next day. 
Obsolescence of knowledge is a fact of life that must always be 
kept in mind, but this does not render futile the assimilation of 
the current content of knowledge or counsel exclusive attention 
to the process whereby knowledge is discovered. As will be 
pointed out below, the availability in learners' cognitive 
structures of broad, over-arching explanatory concepts and 
principles in a given discipline is the most important single 
variable determining the outcome of both meaningful reception and 
meaningful discovery learning. 

A related argument in this context invokes the allegedly 
rapid rate of forgetting of school learning. Actually, however, 
when potentially meaningful subject-matter is optimally 
programmed in accordance with such pedagogic principles as 
progressive differentiation, integrative reconciliation, 
sequential dependency, differential practice schedules, 
overlear ning, consolidation, individualization of instruction, 
immediate and explanatory corrective feedback pr ox ima teness of 
relevance, etc., meaningful reception learning in classroom 
settings, contrary to prevailing educational opinion, exhibits 
impressive longevity — ^eyen over a period of years (Tyler, 1930, 
1934; Ward and Davis, 1938). Further, both these long-term 
classroom studies, as well as more traditional labora tor y- type 
research (e.g., Meyer and McConkie, 1973; Handler and Johnson, 
1977; Brown and Smiley, 1977) indicate that the relative memorial 
longevity of concepts in meaningful text tends to vary positively 
as a function of their relative salience and level of 
abstraction. 

An inportant line of research investigation (e.g.. Ring and 
Novak, 1971; Saugstad, 1955) has demonstrated that in solving 
less novel and less difficult application problems, the degree of 
prior problem-solving experience per se contributes much less to 
variance in problem- solving outcome than does the availability, 
clar ity , and stab ili ty in cognitive structure of the par ti- 
cular concepts and principles that are relevant for the solution 
of the problem in question. Apart from certain general intra- 
disciplinary problem-solving capabilities, such as facility in 
hypothesis f ormula tion and testing , in the general str a tegy of 
efficient application, in identifying fruitful approaches that 
minimize costly risk and unnecessary cognitive strain, in 
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using .systematic and economic n^ethods of inquiry, and in- 
maintaining a flexible and meaninyful learning set, general 
training in problem-solving approaches or stra teg ies manifests 
relatively little generality over different intra-discipl inar y 
problem categories ( Cr utchf ield^ 1966; Parnes and Meadow, 1959), 
in addition to being discipline-specif ic* Typically, it also 
exhibits the circumscribed intra-problem lear ning-to-lear n 
transfer, effects characteristic of Harlow'-s (1949 ) original 
research on discrimination problem-solving . 

Additional supportive evidence of the powerful transfer 
effects of abstract verbal knowledge on problem-solving, even in 
motors, perceptual., and puz:2le-type problems, is provided by such 
transfer studies as French (1954), Hendrickson and Schroeder 
(1941) , nil gard et al. (1953), Judd (1902), Katona (1940), 
Over ing and Travers (1966), and Scandura (1966). It should also 
be appreciated that since some general effects of problem-solving 
experience listed above are transferable within a given 
discipline , knowledge of scientific method acquired through 
discovery experience with one class of problems is general izable 
to other intradisciplinary problem areas. Knowledge of 
scientific .method, therefore, need not be acquired from 
pr obrem-sblving exper ierice with every substahtive concept and 
principle within a discipline. It can be transferred not only 
from verbally expressed methodological generalizations derived 
from prior laboratory experience but also from the exposition of 
purely presented methodological principles as well. 

With respect to the appl ica tion of non-exper ientially or 
didactically-acquired classroom knowledge, however, whether 
substantive or methodological, even though such principles are 
highly transferable to particular classes of practical vocational 
problems, the capability of handling a given problem class 
efficiently is greatly enhanced by intensive problem-solving 
experiences in coping with its distinctive features in an applied 
setting. 

In the case of more difficult and novel problems that are 
not ordinarily susceptible to simple application or to limited 
modification of relevant substantive and methodological content 
in cognitive structure by means of rational thinking processes, a 
complex of such cognitive-personality traits as perseverance, 
resourcefulness, flexibility, originality, problem sensitivity^ 
ventin:e.sGroeness, and improvising ability appears to determine 
most of the variance in problem-solving outcome. Common 
experience indicates that such traits are relatively rare — as 
rare as talented practi tioner s in any voca tional or professional 
area, ;both most likely generally determined, for the most part, 
and not •\?tery teachable. For example, there are very few 
genuinely -good diagnosticians -in medicine who can cope suc- 
cessfully ^ith the rare non-garden variety of diagnostic. 
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problems* These relatively rare individuals — perhaps one 
physician in a thousand — are also * identifiable quite early in 
their medical careers , before they have had a great deal of - 
clinical experience. Their colleagues who lack these traits > on 
the other hand^ do not improve appreciably in diagnostic ability 
over the years^ irrespective of how much experience^ instruction, 
and supervision they may have in diagnosi.9. Clinical experience, 
or instruction, in other words is a minimally necessary but not a 
very significant or sufficient factor in producing good 
diagnosticians in medicine. 

Thus, for both the less and more difficult variety of 
vocational or professional problem-solving, it seems to make more 
sense, in terms of transferability to problem-solving ability, 
for the school to focus on the more efficient and meaningful 
transmission of subject-matter content that constitutes most of 
the raw material from which problem solutions are derived {by 
both the more mediocre and the more talented problem solver) than 
to concentrate on teaching problem-solving skills or traits that 
both are genically less generously distributed in the population 
and are also less teachable. And what is true about exceptional 
degrees of problem-solving capability is even more true about 
creativity , " i.e. , the discovery of uniquely significant and 
original new knowledge as opposed to the application of existing 
knowledge to particularly difficult practical problems requiring 
exceptional inferential, eliminative, analytic, or synthetic 
reasoning skills, in addition to the specific cognitive and 
personality traits mentioned above. 

Natural ("Real Life," Experiential) 
versus Structured Settings 

How important is it that substantive or procedural knowledge 
be acquired and practiced in natural ("real-life" or 
experiential) settings? Enthusiastic supporters of discovery 
learning, beginning with the Progressive Education or "Learning 
by Doing" school of thought, as we have already seen, take a 
rather extreme position on this issue, advocating (1) the use of 
unstructured and uncontrived learning situations; (2) direct 
kinds of experience, in a concrete manipulative sense; (3) 
unintentional or nondeliberate learning effort with respect to 
the component skills involved in a total performance context; (4) 
learning by autonomous, unguided discovery; and (5) exposure to 
diversified rather than repetitive experience. It is important 
at the outset, however, to point out that advocates of 
experiential or naturalistic learning need not necessarily take 
an all-or-none position on any of these five issues and that all 
possible degrees of didactic, classroom, or reception learning 
can be used successfully within the general framework of 
experiential or career-oriented education. 
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It is true, of course, (providing that all other factors are 
equal) that learning is enhanced when its relevance for later use 
is perceived as immediate and proximate; when practice, 
application feedback, and reinforcement are ~conten5)oraneous with 
learning and directly accompany the learning phase of component 
and homogeneous ideational elements rather than follow the 
completion of a total heterogeneous package of didactically 
transmitted content after an interval of months or years; and 
when the conditions of practice closely resemble the conditions 
under which the skill or knowledge in question will eventually be 
used. Such learning is also less likely to be monotonous or to 
be forgotten because of long intervals of disuse and is more 
likely to benefit from higher levels of personal interest (ego 
involvement) and motivation. Wholly natural settings, however, 
rarely provide the learning and practice conditions that are both 
necessary and optimal for efficient learning. 

Generally speaking, it is only in the later stages of 
learning in vocational or professional education, after each 
component substantive and procedural skill has been mastered in 
structural practice sessions, that naturalistic "dress 
rehearsals" become feasible. In the first place, wholly 
na tural istic settings depr ive the. lear ner of the facil i ta t ive 
effects of abstract didactic knowledge (acquired rapidly through 
optimally organized reception learning ) on the later learning of 
more highly differentiated substantive or procedural materials 
and on later problem-solving. 

Second, uncontrived and unstructured (i.e., "real-life") 
learning experiences typically fail to include a sufficient 
number of properly spaced reviews and problem applications as 
well as adequate opportunity for differential corrective 
practice , clar if ica tion , and confirmation of particularly 
difficult components. Third, unstructured practice does not 
receive the benefit of skilled pedagogic selection, presenta t ion 
and organization of instructional materials; of careful 
sequencing, spacing, and gradation of difficulty; and of optimal 
balancing of intra-task repetition and intra- and inter-task 
variability. Fourth, most learning effort is enhanced by 
deliberate rather than incidental intention to learn. 

Fifth, the important principle of initial simplification of 
difficult learning tasks for unsophisticated pupils runs counter 
to iihe doctrine of wholly naturalistic or unstructured learning. 
Exposing unsophisticated learners to all of the complexities of 
na tura 1 , "unarranged , " da ta in the labor a tory , or of sub tie 
distinctions and qual if ica t ions in advanced instructional 
materials, is the surest way of confusing and overwhelming them. 
The use of artificial "crutches" and gradation of difficulty, the 
division of complex tasks into' separate component units, and 
slowing down the rate of presentation are common forms of initial 
simplif ica tion of learning tasks. 
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Lastly^ many features of experiential learning programs were 
based on the extrapolation to older children, adolescents, and 
adults of the proposition that the pre-school and primary school, 
child perceive the world in relatively concrete and intuitive 
terms. It is true that the young child requires considerable 
direct experience with multiple tangible exemplars to formulate a 
concept and with many concrete instances of a given set of 
relationships, before he or she can conprehend abstract 
generalizations. Thus an attempt was made to teach factual 
information and intellectual skills through the medium of direct, 
manipulative experience in natural settings rather than through 
verbal expositions. 

In older pupils, however, once a sufficient number of basic 
abstract concepts and transactional terms are acquired and once 
the skill of comprehending and manipulating abstractions without 
the benefit of concrete-empirical props is mastered in one or 
more subject matter areas, new concepts can be acquired from 
verbally presented definitions rather than from direct 
experience, and new propositions can also be understood without 
any reference to concrete exemplars of their component terms. At 
this stage of intellectual development, therefore, direct 
concrete experience is typically helpful only in the beginning 
phase of -exposure to a completely new and unfamiliar discipline. 
Thus in the secondary school and beyond, it may be desirable to 
reverse both the sequential relationship and the relative 
proportions between abstract concepts and concrete data. 

All of these considerations suggest that various 
combinations of didactic and experiential instruction will be 
more effective in professional and vocational education than 
reliance on purely experiential learning. Most of the first year 
or year and one-half of medical education, for example, might be 
spent in learning the more general, explanatory, and interprative 
concepts and principles of the basic medical sciences (such as 
anatomy, physiology, pathology, bacteriology, and pharmacology) 
with only illustrative reference to more detailed content in 
these fields; and as, pointed out above, such highly abstract, 
general, and salient ideas are highly resistive to forgetting. 
More specialized and highly differentiated details in these areas 
could then be profitably postponed until the student receives 
training in the specialized clinical areas of medicine. At this 
latter time he would not only be better .$ble to appreciate their 
relevance for medical practice, and thus be more highly motivated 
to learn and retain them, but would also have less time and 
opportunity to forget them during the interim when no opportunity 
for appl i cat ion ex ists . 

In addition, instead of learning the entire deta iled content 
of each basic science in one total package and then applying 



individual segments piecemeal and haphazardly one or more years 
later, he would have the opportunity to apply each individual 
segment clinically as he learns it didactically, and, thus, 
simultaneously to receive immediate and proximate feedback after 
didactic instruction in each particular segment as he comes to 
it. 

In training paraprof essionals, in contrast to professionals, 
greater relative emphasis would be placed on the experiential as 
opposed to the didactic component of instruction. For example, 
since a physician's assistant deals with less complex aspects of 
diagnosis and therapy than the physician himself, he/she has less 
need for high-level theoretical sophistication and needs only to 
learn those more general aspects of theory that are immediately 
and proximately relevant for relatively simple, practical 
considerations. It has been demonstrated, for exairple, that when 
a mere summary of the underlying physiology and pathology of the 
endocrinology of pubescence is presented immediately before the 
clinical disorders, the la tter ma ter ial is learned just as well 
as when a much more detailed discussion of the physiology and 
pathology of the endocrinology of pubescence is learned, 
reviewed, and consolidated by several weekly readings prior to 
the learning • of the clinical ma ter ials ( Ausubel and Youssef , 
1966). Paraprof essionals in medicine can thus be trained much 
more rapidly than physicians by dispensing with much of the more 
tangential and less salient aspects of pre-medical and 
pre-clinical medical science education. 

Explanation of Preliminary Findings of 
Lear ning- in-Work Research Program 

The Importance of Long-Term 
Retention Measures 

The methodological decision of the Canter researchers to use 
a long-term retention dimension in ascertaining and evaluating 
the relative efficacy of experiential versus traditional 
classroom learning environments was extremely crucial because, as 
the researchers point out, the traditional pre- test/post- test 
control group design "may restrict the time required for 
differences to emerge" (Crowe and Harvey, 1980, p. 7). Over 
shor t- term retention intervals , compara tive test scores for the 
two groups are misleading because rote retention (e.g., 
short-term cramming) may be confounded with the meaningful 
retention that is more likely to be an outcome of experiential 
"hands on" learning, thereby obscuring one of the principal 
advantages of the latter approach to learning. Also, over the 
long haul, the artifactual test disadvantages for the 
learning- in-work group of using a test instrument designed to 
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measure the learning outcomes of traditional classroom teaching 
are much more likely to be attenuated. 

^ The methodological advantages of long-term retention 
designs^ however^ are in my opinion^ largely negated in this 
context by the researchers* explanation of forgetting in terms of 
retroactive and proactive interference^ principally the former^ 
that tend to have explanatory value for rote rather than for the 
kinds of meaningf ul'lear ning and retention that supposedly 
characterize experiential learning environments. A theory of 
meaningful learning as the acquisition of new meanings (resulting 
from the interaction between potentially meaningful material and 
relevant subsumer s or anchor ing ideas in cognitive structure) , 
accounts for forgetting in terms of the gradual loss of 
dissoc iabil i ty of these new meanings from the more general and 
less qualified meaningful iitport of the more stable anchoring 
ideas that subsume them. The functional incorporation of these 
newly learned meanings into a network of relevant anchoring ideas 
in cognitive structure obviously tends to protect them from the 
proactive and retr oac tive interf er ing effects of similar out 
conflictive stimuli or responses that are so damaging in the rote 
learning and retention of discrete units of learning material 
because in the latter instance they are not nonarbitrar ily and 
nonverba timly rela table to relevant ideas in the learner's 
existing structure of knowledge. 

Most research on forgetting^ suggests that retroactive 
or proactive interference occurs in the forgetting of potentially 
meaningful learning material only when verbatim recall of the 
latter is required or when the learner employs a rote learning 
set. When it does occur in meaningful learning and retention^ it 
is not a reflection of pairing similar stimuli with different 
responses — because students are not typically taught different 
answers to the same question in classroom learning — but rather 
because the interpolated or previously learned material engenders 
cognitive confusion^ ambiguity^ or loss of discr iminabil ity 
between the anchoring ideas and the new meanings derived from and 
stored in relation to them. In fact^ interpolated material that 
enhances these latter effects tends to induce retroactive 
facilitation (Ausubel^ Robbins^ and Blake^ 1957; Ausubel/ Stager^ 
and Ga ite r 196 8) despite the negative influence of similar ity 
short of identity. We shall return to this problem later in 
discussing the retroactive interference explanation of the 
experimental findings. 

Another theoretical difficulty in interpreting long-term 
measures of retention in experiential and other kinds of learning 
is the use of such notions as "long- and short-term memory" as 
explanatory rather than as the descriptive concepts they really 
are. These terms merely descr ibe the temporal / 1 imi ting 
parameters and sequences involved in the processing and storage 
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of information rather than explain the underlying psychological 
processes or mechanisms involved. Thus, for example, meaningful 
learning processes have much more explanatory value for long-term 
memory than vice versa. 

Finally/ we must consider the theoretical confusion 
engendered by using and differentiating between such phenomena as 
knowledge and comprehension in terms of Bloom's purely arbitrary 
and theoretically inconsistent taxonomy of educational 
objectives. By defining knowlecJge as simply "the recalling of 
factual information," the essential characteristic of knowledge 
as meaningfully acquired information is conpletely obscured; and 
ia def ining compr ehens ion as "understanding information without 
relating it to other information," the very process underlying 
comprehension or understanding, namely, relating new potential 
meanings to already existing knowledge in some sensible fashion, 
is similarly ignored, as well as the distinction between 
understanding previously acquired (perception) and new meanings 
(cognitive learning). 

Meaningful retention scores can also never exceed learning 
scores (except for the phenomenon of reminiscence occurr ing 

wi thin severa 1 days following immediate testing for meaningful 

learning) in the absence of intervening practice, transfer, or 
rehearsal. For this reason the occurrence of an incr ea s ing slope 
of the retention- curve in the learning-in-work group, immersed in 
a similarly interfering work environment over the summer recess 
(when no active efforts were made to integrate the summer work 
experience with subject matter knowledge), suggests that it may 
be artifactual, i.e., reflective of sampling, experimental or 
test error. This is especially true when we consider (1) that 
this same work experience during the academic year, accompanied 
by concomitant integration with school knowledge in mathematics, 
yields a learning curve with a nega t ive slope and (2) that there 
is no difference between the "math" and "no math" subgroups with 
respect to this declining learning curve, as there should have 
been if retroactive interference were operative during the 
learning period. 

Additionally, interpreting meaningful versus rote learning 
curves, it is necessary to appreciate that the superiority of 
meaning f ul over rote retention is not simply due to the fact 
that, as the Center researchers suggest, "there is more 
information available to retrieve, since meaningful material is 
learned better or ig inally " ( Crowe and Harvey, 19 80 , p . 35 ) . 
Unlike the occurrence of superior rote retention under conditions 
of superior rote learning, where all of the increased retention 
is attributable to the increased learning (Postman and Rau, 
1957), the same conditions responsible for superior meaningful 
learning also continue to operate during the retention interval 
and, thus, to enhance retention independently of their effect on 
learning. 
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Hence, if such factors as greater meaningf ulness or greater 
frequency, salience, and immediacy of feedback in the 
learning- in-work environment eventuate in greater learning, they 
will also independently induce greater retention as well, over 
and above that attributable to superior learning, with resulting 
divergence of the retention curves for the two learning 
environment groups, after completion of learning phase of the 
learning-retention curve. 



The Retroactive Interference Explana tion 

It is theoretically and empirically improbable, in my 
opinion, both on the theoretical grounds considered above and in 
the face of the overall weight of the research evidence against a 
retroactive interference model of meaningful forgetting (at least 
with respect to the simpler stimulus generalization or stimulus 
or response competition versions of this model) that one can 
account for the findings of the Center study in terms of an R.I. 
paradigm of forgetting which is theoretically and empirically 
credible only in instances of rote learning and forgetting. The 
R.I. explanation also does not fit the implausible finding of an 
increasing slope of the retention curve over the summer recess 
for the learning-in-work group — even assuming the "unlearning," 
"integration," and "fading effects" of the interference of the 
rules for learning to learn in a work environment with the rules 
for learning to learn in a school environment (Crowe & Harvey, 
1980, pp. .138-41). The type of interference that is typically 
involved in an R.I. paradigm is reflective of similarity of 
stimulus or response content in simple and discrete S-S or S-R 
rote associations or of the perseveration of unadaptive learning 
sets in apparently similar but nevertheless conflicting problem 
s i tua tions . 

It therefore seems somewhat far-fetched to suppose that mere 
shifting from the rules of learning to learn in a school 
environment to corr espond ing rules in a work environment would 
generate sufficient interterence to result in impaired responses 
to test items designed to measure learning in the former- 
environment. This is especially the case when one considers the 
compensatory advantages in meaningf ulness and in the various 
dimensions of reinforcement through feedback provided by the work 
environment. At the very most one might reasonably anticipate a 
relatively slower rate of learning as a result of the shift in 
the learning environment rather than an apparent decrement in 
learning score. Seemingly more plausible explanations, 
therefore, would be (1) the anologically relatively poorer scores 
to be expected of students enrolled in one of the new mathematics 
or science curricula^ on tests designed to measure knowledge of 
the traditional curriculum (i.e^^r test disadvantage) and (2) the 
" ar tifactually higher scores that are possible on a rote memory 
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basis (over a short retention interval) when the same traditional 
measuring instruments are used to measure learning outcomes for 
students continuing to learn in a traditional as opposed to those 
who are shifted to a new and unfamiliar learning environment. 
This would explain both the ar tifac tually higher learning scores 
of the traditional group over the short learning (cramming 
testing interval in May as well as the actual steeper decrement 
in retention for this same traditional group over the long summer 
recess retention interval. The test disadvantage explanation^ 
however, could not account for the increasing slope of the 
retention curve in the learning-in-work group, . because the same 
test disadvantage applies in the post-summer follow-up test as in 
the May post-test. 

However, the inadequacy of the test disadvantage explanation 
of the latter finding is discounted somewhat by the strong 
possibility that it is reflective of error, rather than genuine 
treatment variance, and would probably not be replicated with a 
larger and more representative sample and under better-controlled 
experimental conditions* As pointed out above, the maximum 
effect that could be theoretically expected from a continuing 
learning-in-work environment is a more gradually declining 
retention curve, vis-a-vis the traditional learning groups, 
rather than an apparent increase in retention after the long 
summer recess interval during which no obvious practice, 
rehearsal, or transfer effects are operative. 



Greater Meaning fulness and More Effective 
Reinforcement of Lear ning-in-Wor k Environment 

As indicated earlier, learning-in-work environments are more 
likely to result in meaningful learning and retention for several 
reasons. Most important, the very fact that the knowledge is 
acquired in relation to particular, naturalistic problem-solving 
situations and is also performed "hands on" invests the learning 
with both greater proximate relevance and vividness. Such 
learnings are also easier to learn because the problem situation 
provides immediate, perceptible exemplification of the concepts 
or principles involved. 

As emphasized above, although actual application is both 
necessary and helpful, in learning to apply didactically acquired 
knowledge in "real life" vocational situations and is probably 
more meaningful when acquired in such learning environments, this 
explanatory generalization must be qualified by at least three 
impor tant consider a tions . Fir st , it does not imply tha t the 
didactic knowledge itself , that is, the knowledge to be applied, 
must necessarily be the product of either autonomous discovery 
learning or of inductive problem-solving experience. Even within 
the general context of a learning- in-work environment, the 
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underlying concepts and principles can be taught most ettectively 
by expository teaching methods and acquired most efficiently 
through meaningful reception learning* m practice ^ this 
consideration is followed by most vocational training programs r 
both civilian and military^ including the Center learning study. 
That is^ classroom instruction is utilized as an integral part of 
the general experiential learning setting* Research^ has 
also shown that a guided discovery approach is more successful in 
applying general principles to particular problems than either 
completely autonomous discovery or completely guided problem 
solving. 

Second , educators in learning-in-work settings must be 
disabused of the deeply rooted belief in our educational folk 
lore that practical problem-solving experience in an experiential 
learning environment necessarily and inevitably leads to 
meaningful learning and retention. Actually it is quite commonly 
the case that on-the-job trainees simply imitate their preceptors 
blindly or memorize procedural steps in problem solving without 
having the faintest idea of what they are doing and why. Such 
experiential problem-solving experience is obviously just as rote 
as rote memorization of scientific formulas or of geometrical 
theorems r without understanding the principles involved, and is 
equally not transferable either to related problems or even to 
the same problem couched in slightly different language. What 
over-enthusiastic proponents of "learning by doing" apparently 
overlook is that one does not learn anything by doing unless one 
understands what one is doing and why. 

Still a third misconception , often attributed to John Dewey , 
but actually advanced and popularized by some of his more 
overzealous disciples, is the notion that knowledge gained 
through direct exposure to concrete experience is invariably more 
meaningful and transferable than when acquired through abstract 
verbal instruction in the classroom. This idea may often be 
true, but for reasons that have nothing to do with the 
abstractness or concreteness of the learning experience per se, 
once learners have either developed generally beyond the concrete 
stage of logical operations or, if generally beyond this stage, 
are no longer confronted by the initial task of learning the 
vocabulary of an entirely new discipline. It is rather a 
reflection of the fact that knowledge acquired in "real-life" 
learning environments has greater proximate, immediate, 
perceptible, and personalized relevance than the same knowledge 
acquired in a classroom, is typically more individualized, and 
also does not need to be transferred to the context in which it 
will ultimately be used. Counterbalancing these advantages, 
however, are such factors as the fact that structured classroom 
learning can profit more from initial simplification and 
progressive gradation in difficulty as well as from skillful 
pedagogic programming, sequencing, pacing, and consolidation, and 
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is also typically characterized by more systematic and 
differential practice schedules. 



Reinforcement interpretations of the greater learning and 
retention to be expected in learning-in-work situations are 
probably valid insofar as they invoke such dimensions of 
reinforcement (feedback) as frequency ^ immediacy, completeness, 
relevance f vividness, and individualization. We have already 
discounted the importance ot the abstract-concrete dimension. In 
addition, reinforcement is probably more effective when provided 
immediately through application experience after each component 
segment of a course or curriculum than when all didactic 
instruction is giv3n first in one large block of time comprising 
months or years and is then followed by practical application in 
"real-lite" situations in a comparably large block of time, as is 
typically the case in such professions as medicine, dentistry, 
law, etc. All of the reinforcement dimensions of frequency, 
immediacy, and proximateness of relevance favor the. latter 
experiential over the former traditional type of professional 
education. Furthermore, the former organization of the 
curriculum provides greater motivation to learn and less 
forgetting when the practical relevance of each didactic segment 
. fpr_ prof ess ional_pr act ice is^^^p and applied immedi ately 

than when practical naturalistic experience is mostly provided 
after the completion of all preclinical courses. For this reason 
we have advocated the deferral of all detailed, anatomical, 
physiological, pathological, and pharmacological instruction in 
medical schools until such time as relevant clinical specialty 
courses and actual naturalistic problem-solving experiences are 
provided (Ausubel, 1980). 

From a theoretical standpoint it is also important to 
recognize that reinforcement in meaningful learning and retention 
does not operate in accordance with the law of effect, whereby 
the (habit) strength or probability of recurrence of a response 
(instrumental or rote) to a particular stimulus, or simply of an 
emitted response, is a function of- the satisfying consequences of 
the response in question. Knowledge of results, or the 
et f ect iveness ot feedback regarding the success or failure ot the 
learning effort, serves as cognitive reinforcement (by providing 
confirmation, clarification, and correction) and also enhances 
the existing motivation energizing this effort (i.e., by 
increasing attention, self-confidence, perseverance, etc.). 

Even in instrumental, operant, and .ro^te learning, the habit 
or availability strength explanation of reinforcement is not very 
convincing. More credible, and also more congruent with innately 
( genie ally ) determined hierarchies of probabilities ot 
instrumental responses when a given drive state is operative, is 
the explanation that reinforcement, under instrumental or rote 
learning conditions, increases acquired response probability by 
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lower ing response or perceptual" thresholds rather than by 
increasing habit or availability strength . This explanation is 
also more consistent with fluctuations in response probability in 
the opposite direction (repression, blocking , or response 
competition and "learning shock"); with such d isinh ib i tion 
phenomena as reminiscence and increased recall under hypnosis; 
and with the differential availability of rote or anecdotal 
memories in recognition versus recall criteria of retention 
(Ausubel, 1968, pp. 104-05; 384-86; 391-93). 

In meaningful learning and retention, however, since new 
meanings are invariably part of a larger subsystem of ideas in 
the learner's structure of knowledge, rather than discrete and 
isolated S-R connections, it is theoretically improbable, as well 
as empirically undemons trable , that satisfying effects of 
feedback can lower thresholds of availability. It is much more 
likely that motivational and effective factors (anxiety or guilt; 
negative attitudinal bias) negatively influence the retrieval of 
learned meanings by ra ising thresholds of availability (repres- 
sion) than that positive attitudinal bias lowers thresholds of 
availability. Meaningful retention, as pointed out above, is 
also influenced by the same cognitive structure variables that 
influence meaningful learning and then continue to affect 
retention in the sanie direction during the retention interval. 

Finally, it seems theoretically unwarranted, in my opinion, 
to suppose that the perception of greater task complexity or of 
greater internal versus external control by members of the 
learning- in-work group is reflective of higher intrinsic 
motivation. Intrinsic motivation, by definition, connotes 
motivation to^lear n^-as an end in itself, i.e., for the 
satisfaction of knowing, understanding, or being able to master 
the challenge of problem situations rather than for some 
utilitarian, career-oriented, or ego-en.'-;anc ing purpose. Hence, 
there is no good reason for believing that learning by problem 
solving for car eer--or iented purposes in an experiential, 
naturalistic setting is inherently any more intrinsically 
motivated or any less energized by ego-enhancement considerations 
than rote or meaningful reception learning for the purpose of 
obtaining high grades in a traditional classroom setting. In 
both cases the type of motivation—intrinsic or extr insic-- tha t 
is primarily operative can only be ascertained by motivational 
and personality analysis of the particular learner in the 
particular learning environment rather than by formal membership 
in a particular learning environment group or by the reception 
versus the discovery approach to learning used in that learning 
environment. 



E-23 



ERLC 



Suggested Changes 



in 



Research Design and 



Ob jec tives 



Apart from such obvious methodological changes as increasing 
the size and representativeness of the sample, random assignment 
of subjects to experimental and control groups, and eliminating 
or counterbalancing, as far as possible, test disadvantage for 
both groups, perhaps the two most important changes that are 
required in research design are (1) overcoming the massive 
Hawthorne effect and (2) avoiding the imprecision and ambiguity 
inherent in constituting experimental and control groups in terms 
of such global and unspecif iable terms as "traditional" and 
"learning-in-work." Not only are these latter criteria vague and 
susceptible to almost infinite variability, but the precise 
independent var iables responsible for ob ta ined differences 
between the two groups in learning and retention outcomes are 
also neither specified nor controlled and their relative 
contributions to the differences in outcomes are indeterminable. 
It will be recalled that the curriculum reform movements largely 
foundered for similar irremediable defects in measurement and 
. evaluation involving (1) identification of the large number of 
independent variables involved and their respective 
indeterminable relative effects on learning and retention 
outcomes and (2.) the virtual impossibility of controlling for the 
massive Hawthorne effect under such experimental circumstances.^ 
Both difficulties could be simultaneously overcome, in my 
opinion, by adopting a multivariate instead of a global and 
unspecified experimental-control group research design. 

r^uch a multivariate research design would presumably be 
capable of identifying and measuring precisely both the learning 
and retention effects attributable to a host of independent 
variables and the various interactions between them, as well as 
of differentiating between different levels of learning, 
retention, transfer, and problem— solving outcomes appropriate for 
vocational, paraprof essional , and professional training, 
respectively. In no other way, also, would it be possible to 
control for such highlyXconsp icuous Hawthorne effects resulting 
from the use of a lear ning-in-wbr k versus a traditional classroom 
research design. 

In very brief outline form, the following main categories of 
independent variables should be systematically varied in 
different multivariate research design studies: (1) Cognitive 
Str uc ture : a va ilab il i ty , pr ox ima teness of relevance, clar i ty , 
stability, and discr iminab il i ty of anchoring ideas; use of 
expository and comparative advance organizers; progressive 
dif f er entia tion, integra ti ve reconcil ia t ion, sequential 
dependency, and consolidation in programming subject matter; 
availability of relevant background ideas in students' cognitive 
structures versus teaching of problem-sol ving strategies in both 
"application" and "reorganization of background knowledge" types 
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of problem solving; level of total subject matter sophistication; 
cohesiveness and integra tiveness of subject matter in learner's 
structure of knowledge, (2) Aptitude ; general intelligence; 
primary mental abilities; dimensions of cognitive style affecting 
both reception and discovery learning (e^g,, 

par ticular izing-generalizing trends, divergent versus convergent 
thinking ability, integra tiveness versus compar tmental iza tion of 
knowledge, preference for coirplexity versus sinplicity), 
cognitive and personality determinants of creativity. (3) 
Motivational ; intrinsic versus extrinsic; task-oriented versus 
ego-oriented; cognitive drive; affiliative drive; level of 
achievement motivation. (4) Per sonal ity ; satellizer versus 
non-sa tell izer ; dogmatism; closed- versus openmindness ; tolerance 
for ambiguity; test and personality anxiety; locus of control; 
inner- versus outer-dir ectedness . (5) Overall developmental 
stage of cognitive functioning. (6) Task or Treatment Variables 
(practice and instructional materials): frequency of practice; 
distribution of practice; difficulty and quantity of learning 
material; pacing; individualization; linear versus differential 
practice schedules; mastery learning; learning set and warm-up 
variables; homogeneity versus heterogeneity of learning tasks; 
single-context versus mul ticontex tual concept and rule learning; 
structured versus incidental learning; different mixes of 
didactic and experiential learning in terms of proportion, 
relative emphasis, sequency, whole versus part 
didactic-application strategy; naturalistic versus contrived 
learning setting s; transfer versus direct practice; verbalization 
of insight; abstract versus concrete exemplification of concepts 
and principles. (7) Re inf or cement : completeness, frequency, 
immediacy, invariable versus intermittent; corrective versus 
confirmatory. This listing, of course, is intended to be 
illustrative rather than exhaustive in nature. 
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Footnotes 



1 For a review of studies in this area see Ausubel (1968 
pp. 115-18; 1977, pp. 171-72). 

2 For references demonstrating lower final post- test 
scores for students enrolled in new mathematics or science 
curricula on tests measuring knowledge of the traditional 
curriculum ma ter ial and the opposite effect for students enrolled 
in traditional curricula on post- tests measuring knowledge of the 
new curriculum content, see Ausubel (196 8 pp. 357-5 8). The latter 
effect, of course, is largely just as reflective of test 
disadvantage for. the traditionally instructed students as the 
former effect is reflective .of test disadvantage for the 
innovatively instructed students (since, additionally, there is 
strong reason to believe that much of the material in these new 
curricula were rotely rather than meaningfully learned because it 
was too difficult in terms of the students' existing degree of 
subject-matter sophistication latter explanation. This would be 
somewhat more true, for example, of the BSCS (Ausubel, 1966) than 
of the PS'C (Finlay, 1959 , 1960) or UICSM (Beberman, 1958) 
curricula. 

3 For a summary of research in this area see Ausubel 
(1968, pp. 302-05) . 
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This paper is an examination of the report entitled 
Retention of Concepts Resulting from Learning by Experience by 
M.R. Crow and R.J. Harvey. The approach of the paper is to 
critically examine the results and the theoretical model proposed 
to account for the findings, to examine proposed alternatives, to 
suggest new interpretations, to suggest new research 
possibilities, and to propose ways of improving the current 
research design. 

Comments on Theoretical Interpretation s : 
Inter f er ence 

The principal surprising result of this study was the loss 
in-ma"th test performance over the nine months' instructional 
interval f or . the learning- in-work-environment students, coupled 
with a gain in performance for this group over the three-months' 
retention interval. The modest loss in math test performance 
during the instructional interval is perhaps not too surprising; 
however, the gain in math performance during the retention 
interval does represent an anomaly and is certainly an 
unanticipated finding. 

In contrast, the gain in math performance during the 
instructional period followed by forge tting dur ing the retention 
interval is consistent with previous findings on traditional 
classroom groups. 

The authors propose an interference/assimilation model to 
account for the anomalous findings. The basic issue is to 
interpret, consistently, the decrease in math test performance 
during the instructional interval and the increase during the 
retention interval for the learning-in-work-environment students. 
Retroactive interference is used to account for the loss during 
the instructional period. Here it is assumed that students in 
the learning-in-work environment were acquiring new skills and 
concepts (during instruction) which interfere with prior learning 
acquired in a traditional formal school environment. This inter- 
pretation is quite reasonable and logically follows" from viewing 
the instructional sequence as a retroactive interference 
parad igm. 

But this interpretation leaves out one important feature of 
this paradigm. It is also quite possible that the loss (forget- 
ting) that occurs during the instructional interval is due to 
other more non-specific interference. The design does not have a 
control that is comparable to one found in the typical labora- 
tory study of retroactive interference. Usually, a control group 
which learns some "unrelated task" is compared with the 
experimental (treatment) group. Thus any loss in performance can 
be actually attributed to the treatment itself. But a comparable 

F-5 



ISO 



control does not exist here, which can be appreciated by 
considering the following, question: Specifically^ what v/ould 
happen to a hypothetical control group that engaged in some 
wholly unrelated activity during the retention interval? 
(Otherwise they are tested at all three times.) If they showed a 
loss comparable to the learning-in-work group^ then the 
forgetting shown by the learning-in-work group could not be 
attributed to learning specific interfering rules or concepts. 

Of course^ it does not seem practical to have such a 
control. But it is important to consider this; issue. If the 
presumed interference is more general and not specific to the 
features of the work environment^ then research focusing only on 
specific interference could be misleading. 

An alternative way to approach this issue is to make the 
radical assumption of no interference . Assume that the skills 
and knowledges learned in the learning-in-work environment have 
no interfering effects on prior school learning.* In effect^ 
assume that these represent two reasonably independent systems of 
skills and knowledges^ and that they neither interfere nor aid 
each other. How then would one account for the loss in math 
performance ot the learning-in-work group? 

A reasonable candidate is simply that t-he learning-in-work 
students receive insufficient practice on their math skills to 
main.tain the level of knowledge they had nine months earlier. 
Forgetting occurs but is not due to interference specific to the 
skills learned in the work environment. 

I will add that I prefer the specific interference interpre- 
tation proposed in the report. However, I believe that it is 
important to consider and reject these alternatives before 
accepting the specific interference hypothesis. The point is 
that you would be ill-advised to pursue an extensive research 
program based on interference assumptions without considering the 
alternatives . 

Finally^ it might be worthwhile ,to consider designing a 
small-scale laboratory study of interference in learning math 
concepts using an analogue study. The basic design would first 
teach students some new math concepts, then split the students 
into three groups r one a control learning new math, the second an 
experimental group learning something like what the learning- 
in-work students learn, and a third learning some unrelated task, 
followed by a retention test of the initially learned math 
concepts. Unless you could demonstrate interference in this 
laboratory setting, you would have much less confidence in 
pursuing this interpretation. Offhand, I am unaware of any 
studies on interference in retention of math concepts but a 
survey of this literature may show some support. 
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Theoretical Interpretations : 
Assimilation 

The second explanatory construct proposed is assimilation of 
information. The assumption is , that during the summer retention 
interval the iearning-in-work students assimilated the new rules 
and concepts learned during the instructional period with 
existing knowledge structures. This assimilation led to a new 
cognitive structure which provided for enhanced performance on 
the retention test. 

This interpretation is intuitively appealing but is faced 
with at least two difficulties. First, the account is not 
terribly parsimonious. One could simply follow through with the 
retroactive interference argument , and use the fact that the 
7~ amount of retroactive inhibition tends to decrease as the time 
interval between the second-task learning and the test of 
retention increases. Simply as an empirical finding {cf. 
Underwood, 1948), known to occur over short periods, this 
progressive decrease in retroactive interference would account 
for the improved performance on the retention test. Moreover, 
continued improvement at later testings would not necessarily 
support an assimilation argument, I have no objection to the 
assimilation argument per se, but it may not be necessary. 

A second problem with the assimilation interpretation is the 
implicit assumption that it begins and/or is most active during 
the summer session. It is necessary to make this assumption to 
account for the pattern ot learning/retention findings. But it 
could be ar^;ued that assimilation starts much earlier, maybe 
after two to five months of instruction. More generally, 
whenever one has to postulate two processes (in this case, 
interference and assimilation) which generate a U-shaped func- 
tion, it is difficult to evaluate the interpretation. Unless one 
can independently specify when the process of assimilation 
begins, there is no way to predict with accuracy when retention 
should improve. 

Criterion Task: The CTBS 

As noted in the major report, the question of content 
validity of the CTBS may be seriously raised. The appropriate- 
ness of the test as an instrument for evaluating the effects of 
nontradi tional learning can be questioned. The criterion issue 
is frequently complicated by other issues. If all educational 
innovation is to be evaluated by scholastic achievement tests, 
then one will frequently, I suspect, be unable to show much 
effect ot novel educational environments. Alternative tests 
which sample problem-solving skills, ability to generalize to new 
situations, etc. may be more appropriate for evaluation 
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purposes. At best I would suggest the use of several other 
measures beyond the CTBS . 

Indeed/ if there are genuine albeit subtle effects due to 
learning-in-work environments ^ it will be necessary to consider a 
range of instruments that might tap these effects. 

Implications of Learning-in-Work Environments ; 
Generation Effect and Variability Effect 

In principle f there are several features of any idealized 
learning-in-work environment which should produce reasonable 
gains in achievement. The EBCE approach should benefit students 
for the reasons cited in the report^ but it should also benefit 
students for at least two additional reasons. 

There are two features of the EBCE program which are 
directly relevant to contemporary psychological research in human 
memory; One feature is that in the EBCE program students learn 
to generate many of their own solutions to problems. Recently ^ 
Slamecka and Graf (1978) have provided evidence for what they 
call the generat.ion effect in recall. The basic finding in a 
series of exper indents is that if subjects have to generate an 
answer rather than simply remember one, recall is substantially 
improved. Jacoby (1978) has presented similar findings. He 
reports that if subjects have to solve a problem rather than 
remember a solution ^ recall is enhanced. These 'findings appear 
consistent with the older notion of learning-by-discovery and it 
would appear that the opportunities for discovery learning are 
substantially greater in the learning-in-work environment. 

It would appear that a fruitful approach . would be to examine 
the EBCE program in detail to see to what extent generation or 
discovery principles are present. Moreover / these could be 
introduced as part of the program package. Because of the j> 
renewed interest in the generation effect in memory , I would 
recommend an examination of this process and that experimentation 
dealing with it be introduced in the EBCE program. 

A second feature of the EBCE program is that it provides for 
a variety of experiences not found in the traditional classroom. 
Classic studies (e.g.^ Duncan^ 1958; Morrisett and Hovland, 1959) 
have demonstrated the importance of task variety in both transfer 
and problem solving. In both studies ^ performance improved as a 
function of increased variation in the tasks. Recent studies 
have also shown the importance of stimulus variability in memory 
(cf./ Dukes and Bevan^ 1967; Ellis / Parente / and Walker / 1974; 
Ellis^ Parente, Grah, and Spiering^ 1975; Cosden^ Ellis^ and 
Feeney^ 1979). In all of the studies by Ellis and his 
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colleagues, subjects were presented with a perceptual grouping 
task consisting of letter strings which masked the meaningful 
structure of the material. (When properly grouped, the letters 
combined into words or sentences.) The letters were presented in 
constant or varied groupings, and subjects recalled the varied 
groupings more frequently. The superiority of varied input is 
known as the var iab il i ty effect in recall. In other words, 
seeing the information to be remembered in varied fashion aided 
recall because subjects were more likely to chunk the information 
into meaningful groupings. 

It would also appear fruitful to examine the EBCE program 
from the perspective of determining the range of varied tasks or 
stimuli present. In addition, it would be potentially useful to 
design a program with some predetermined built-in variability. 

More generally, it is proposed that two basic elements, at 
least theoretically, of the EBCE program are opportunities for 
discovery ar:.d task or stimulus variation. Research designed to 
directly examine these two features in the context of 
learning-in-work environments should be potentially fruitful. If 
these elements are truly present, then benefits should result. 



Implications of Learning-in-Work Environments ; 
Depth of Processing and Cognitive Effort 

Two fairlly recent developments in human memory may have 
broad implications for this research. One is the depth of 
processing framework and the other is the notion of cognitive 
ef f or t . An important idea in human memory is the depth or levels 
of processing concept. In its original form, the basic notion 
was based on the premise that a series of analyzer .s, . vary ing 
along a continuum from structural to semantic analysis, is 
employed in processing an item for storage in memory (Craik and 
Lockhart, 1972). Expansion of this idea was later made by 
inclusion of elaboration and breadth of processing (Craik and 
Tulving, 1975) and distinctiveness (cf, Cermak and Craik, 1979). 
Although there are logical problems with the concept (e.g.. 
Nelson, 19 77; Baddeley , 19 78) , the notion may have some 
usefulness here. One possibility is that students in the 
learning-in-work environment fail to process the information they 
have learned at the same depth as the controls. Another is that 
more semantic conceptual processing occurs later during the 
summer period. (This latter is not unlike the assimilation 
construct. ) 

The other concept is cognitive effort. Recent studies have 
shown that retention of information is a function of the degree 
of cognitive effort present during study (Tyler, Hertel, 
McCallum, and Ellis, 1979; Eysenck and Eysenck, 1979). Tyler 
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et al. define cognitive or mental effort as the amount of the 
available processing capacity of the limited-capacity central 
processor utilized in performing an information-processing task. 
Although the implications of this concept are not fully apparent^ 
it is proposed that it be discussed by the panel. One 
possibility is tha t ' sub jec ts in the learning-in-work do not 
process the material as intensely as the controls* 

Implications of Lear ning- in-Work Environments : 
Individual Differences 

The role of individual differences^ while not neglected in 
this study^ is an area which could be explored in greater depth. 
The domain of personality factors^ cognitive styles ^ mood- 
emotional states^ and motivational variables should be carefully 
considered in future research on learning-in-work settings. The 
use of any of these factors would serve the double advantage of 
increasing the precision of any experiment when used^ say^ as a 
levels factor y and would provide information in its own right. 
Renewed interest in individual differences is attested by the 
fact that the entire issue of the December 1979 issue of the 
Journal of Research in Personality was devoted to research on 
individual differences in human learning and memory (e.g,^ 
Battig, 1979? Cosden, Ellis, and Feeney, 1979? Eysenck, 1979; 
MacLeod, 1979; and Schwartz, 1979). 



Instructional Considera t ions . 

The only point I wish to consider here is the instructional 
balance in the ECBE program. Is four days of in-work training 
and one day of classroom training the optimal balance? Would a 
two- three day combination (or some other) be better? 



Transfer Con s idera t ions 

Another way of viewing the research is to consider the 
learning sequences from the viewpoint of a transfer paradigm* 
This is not antithetical to interference considerations but 
represents merely another aspect of the sequence. Examination o 
figure 38 (p. 139) of the model will make this clear. Events A 
and B represent two successive tasks in which transfer effects o 
A to B may occur. Ten years of learning in a formal school 
environment (A) should continue to transfer positively to more o 
the same. In contrast, transfer to the activities in a work 
setting may be negligible or yield even negative effects. 

I would like to discuss any possible implications of this 
perspective. 



Design Consider a tfions 



In this section I will briefly comment on a few design 
considerations. The emphasis will be on how to improve research 
design, for future studies. 

1. Tho authors are aware that a basic problem in the 
design is that subjects in the EBCE conditions were not randomly 
assigned. father, students selected this program. This is 
understandable since it vould be extremely difficult to have 
arranged for random assignment. Although subjects were matched 
on sex^ school membership, and GPA, I suspect that there were 
motivational differences between the groups. I would suggest 
that some matching on motivational and/or personality factors be 
considered in future studies. 

2. Most statisticians, I suspect, would argue that 
multivariate analysis of variance would be appropriate (rather 
than univariate analyses) since multiple dependent variables were 
used. ^ 

3. I did not see any correction for the number of 
statistical tests conducted. Given that at least 131 tests of 
significance were run (42 in table 1, 99 reported in table 2), 
the question of some adjustment is raised. 

4. I think future studies should more directly attempt to 
get the student to integrate what is learned in the work 
environment with academic skills and knowledge. 
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Forgetting : 



Some 



Perspectives 



Benton J. Underwood 
Uor thwe s ter n Un i ver s i ty 



Forgetting is the failure to recall or recognize events 
which could be recalled or recognized at some earlier point in 
time. No psychologist is needed to tell us that forgetting can 
be very distressing , pa inf ul , and sometimes lethal . It i s 
difficult to see why forgetting has not been gradually eliminated 
by evolutionary forces over the milleniums. But it has not been 
eliminated and we must accept forgetting as a fundamental char- 
acteristic of living organisms and a phenomenon to be understood. 

There are some general issues that must be identified 
initially before we get into the examination of data which must 
guide our thinking about forgetting. These issues are at various 
levels of discoiarse, are not clearly and systematically related, 
and therefore will sirnply be numbered to reflect this. 



General Issues 

1. Laboratory r -search- on forgetting is nearly 100 years 
old| its beginning commonly being identified with Ebbinghaus 
about 1885. Some may feel that few advances have been made since 
Ebbinghaus, but 1 believe that most of the points to be made in 
this paper tend to deny this. The conceptions of memory and for- 
getting, and the empiricail facts which are most important today, 
are sinply different from those about which Ebbinghaus wrote. 

2 . Forgetting cannot be studied in i sola t ion from 
learning because forgetting is probably in some way (not yet 
understood) determined by the nature of the learning codes. 
Learning of a given task establishes a memory which is generally 
accepted to consist of more than one type of information. Thus, 
when learning is viewed as information processing, the process of 
learning consists of establishing memories which may consist of 
several different types of information. Thus, a memory might 
include temporal information telling when the memory was estab- 
lished ; it might include frequency in for ma t ion, affective 
information, conceptual information, associative information and 
so on. These various types of information are sometimes spoken 
of as memory attributes. If a collection of these attributes is 
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what constitutes a memory--is what results from learning — then 
conceptions of forcjetting must sooner or later deal with the loss 
of these attributes over time. At the present time very little 
progress has been made with respect to this. 

.3 • Different models of memory emphasize different aspects 
of the learning-forgetting system. We may examine two such 
models by way o£ illustration. The first model assumes a system 
with limited capacity^ and further assumes that forgetting is 
adaptive. Given a finite capacity f to establish new memories may 
require that some of the old memories have to be "dumped." This 
model has been illustrated in a story that surely must be famil- 
iar to some. A young biologist was named president of a small 
liberal arts college. Actually f as a biologist he had special- 
ized in £ish, hence he was an ichthyologist. He decided it would 
be a good idea to learn student's names so that he could greet 
them when he met them on the campus. He went about his task with 
great vigor and success until he determined that he would have to 
give up the idea because he found that each time he learned the 
name of a student^ he lost the name of a fish. One could muster 
some evidence for a limited capacity model if a .short-term memory 
system was all that was involved ^ but for long-term memory the 
model seems quite separated from reality. We cannot really note 
any practical limits on the capacity of the long-term memory 
system . 

A second model is based on a most surprising assumption, the 
assumption that there is no forgetting. Nothing that is learned 
is ever forgotten is the basis of this model. Memories/ it would 
be said^ do fade but do not disappear into nothingness. On the 
face of it^ this seems to be an absurd assumption ^ and it may be / 
but the fact is that some respected investigators have accepted 
the assumption as a working hypothesis. Of course the obvious 
question which such a model evokes is what it is that we call 
forgetting. The fact that forgetting occurs appears so obvious 
that it seems ridiculous to assume otherwise. The trick to 
overcome this problem is embedded in a further assumption of the 
model that what we call forgetting is really a retrieval failure, 
and if we look long and hard enough we will find retrieval cues 
which will allow us to recover all memories that have ever been 
established . 

Several coniments should be made about this model. First, 
given this model, research taiies a different direction from that 
traditionally taken because the search will be for an 
understanding of retrieval cues. Second, there is simply no way 
to prove that there is no forgetting. Of course, ^ lis is not 
critical for the model; much could be learned abcc the 
learning-memory system by using this model and whether there is 
truth in it or not is not a heavy burden to carry. Third, some 
interesting data arise from some of the research that comes from 
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emphasizing the importance of retrieval cues. For example^ a 
study was done recently at the University of California at San 
Diego in which observers, several years removed from high school 
graduation, were asked to recall the names of high school 
classmates. The subjects worked at this task for ten hours — an 
hour a day for ten days. The important finding, for our 
purposes, was that even during the tenth hour these subjects were 
still recalling new names. That is, they were recalling names 
they had not recalled earlier but that, according to the 
yearbooks from the high schools, indeed belonged to members of 
the class in question. 

Fourth, the more classical model of forgetting certainly 
does not deny the importance of retrieval cues, but it might ask 
not only how memories are at least momentarily forgotten but also 
how it is that the retrieval cues are forgotten. It has always 
seemed to me that there is no way to avoid the issue that there 
is a phenomenon that is called forgetting that involves the loss 
of "something," and it is our task to try to understand how this 
loss occurs. That is the perspective with which I will proceed 
with this paper. 

4. We must face the fact that most of our systematic data 
on forgetting have come from experiments 'in which word lists have 
been used. I frequently hear the criticism that most of our work 
has used nonsense syllables as the units of study. In relatively 
recent times, say, "from I960, it is a rare experiment in which 
nonsense syllables have been used in the study of forgetting. 
Nevertheless, most experiments have used work lists, and the 
question may be raised as to how general izable such results are. 
It can be stated that most of the phenomena which will be 
described later have been found at various levels of material 
complexity. For example, retroactive inhibition (an interference 
phenomenon) has been found with sentences and with paragraphs as 
the units of study. Heavy interference can be produced with 
hierarchically ordered conceptual systems. Nevertheless, I do 
not know the degree to which a finding might be applicable to 
school subjects, such as ma thematics and reading. I am not 
acquainted with- any systematic work on the retention of school 
subjects. There are many, many problems attending such research 
although there are certainly some issues that could be attached 
with 'benefit. As it is, we can only assume that the laws or 
principles of forgetting which have evolved from work with 
relatively simple materials have some pertinence to the 
forgetting of school subjects. 

5. The role of reinforcement has probably been over- 
emphasized as a factor in learninL,, at least for the young adult 
subject. There have been many studies in which incentive 
variation has been shown to have no influence on learning. There 
are, furthermore, studies which show that immediate knowledge of 
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results (so-called immediate reinforcement) is inferior to 
'delayed knowledge. The implication of these remarks is that in 
the study of forgetting one finds essentially no role for the 
idea of reinforcement. Indeed, it is quite possible to have 
subjects learn as much incidentally as they learn intentionally. 
Since the subjects are not trying to learn in the incidental 
conditions, it is difficult to see how any very direct forms of 
reinforcement can be implicated. 

6. Two measures of learning and memory are commonly used, . 
recall and recognition. There is now a substantial body of 
evidence that indicates that under normal circumstances these two 
indices of memory reflect differences in the nature of the 
attributes (types of information) in memory used by the sub3ects 
on the memory test. Having noted this, we must then say that 
most of the evidence we have on forgetting over long intervals is 
based on the recall measures, so that of necessity we will be 
dealing with recall as an index of memory. 

One can always raise quest'. ons about any response measure in 
any area of investigation. Usua.ly we want the response measure 
to reflect the essence of the behavior in which we are 
interested. However, there are other requirements we impose on 
the response measure, most notably it must be objective and 
capable of being quantified. An observer of our subDects as they 
engage in the various steps of an experiment sometimes sees the 
subjects doing things that seem to reflect directly the behavior 
of interest. However, when we look at the response measure we 
are using we find that nothing happened to support the 
observations. I remember when the people at Yale were first 
working on frustration and aggression before World War II. In 
one experiment they kept their subjects awake all night and 
frequently treated them in beastly ways during the night, e.g., 
they promised the subjects a fine meal but this food never 
arrived. Periodically during the night the investigators took 
objective measures which they presumed could reflect increasing 
aggression as the night wore on. None of these measures showed 
any effects of the treatment. However, the experimenters were 
sure behavior had changed during the night. They pointed to a 
picture drawn by one of the subjects, a picture which showed a 
man being hanged on the gallows, and this man was labeled 
"psychologist!" The picture was taken to indicate clear evidence 
of aggression whereas this aggression was not picked up at all by 
the objective ratings scales that had been used. The point to be 
made is that we must think carefully abouf our response measures. 
The recall measure seems to be a sensitive one, for studying 
forgetting, and one which is ecologically valid and quite 
objective, but we should not use it unthinkingly in all 
s i tuations . 
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The Simple Case 



We will now evaluate the evidence with regard to the simple 
case of forgetting. Ideally, the simple case consists of a naive 
subject who is given a single task to learn, and then retention 
is taken after a day for one group, and after a week for another 
group. The usual finding is a forgetting curve with the amount 
forgotten over 24 hours being at a somewhat faster rate than that 
which occurs between one day and one week. The first question we 
need to ask is what variables will change the rate of forgetting; 
what variables accelerate it. 

Not infrequently we hear experimental psychologists 
justifying their lack of progress as being due to the fact that 
behavior is so complex. It is said that a large number of 
variables influence behavior and these variables frequently 
interact. This situation is not present in the case of the 
simple retention paradigm. Evidence from the early years of 
research clearly showed that. Recently, in an attempt to bring 
this matter up to date, I studied the research published during 
the past ten years and I found that the generalization still 
held: Very few independent variables influence the rate at which 
forgetting occurs in the above-described simple paradigm. Let me 
first indicate some of the variables which do not influence 
forgetting. 



Irrelevant Variables 

All task variables can be included in this category. By a 
task variable I mean any systematic variation in the nature of 
the units used in the task. For example, various forms of 
meaningf ulness represent task variables. We might, for example,- 
compare the forgetting of a list of nonsense syllables with the 
forgetting of a list of common words. We observe a marked 
difference in the rate at which the two lists are learned, bdt 
given that the same level of learning is achieved- for* the two 
types of materials, forgetting rates do not differ. That the 
usual person has believed otherwise indicates that there is a 
confusion between learning rate and forgetting rate. Other task 
variables, which may influence learning markedly, have also been 
shown to have no influence on forgetting. For example, intra- 
list similarity, whether formal, conceptual, or meaningful 
similarity, will produce some marked differences in learning, but 
again, rate of forgetting does not differ. 

A list can be structured so that three different conceptual 
levels are present, and their presence facilitates the learning 
as compared to the case where the same items are unstructured. 
Still, the forgetting of the two tasks will not differ. A 
relatively recent development is known as depth- of -processing 
procedures. The idea is that amount learned of a given task will 
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depend upon the depth to which the items are processed. If the 
subjects were asked to cross out vowels in the words in a list 
the depth is shallow? if they are asked to rate the pleasantness 
of the. words the depth is said to be substantial* The data show 
that recall is higher for the group given the rating task than 
those given the cross-out task, Never theless if learning of the 
two tasks is taken to the same levels forgetting will not differ. 
There have been studies using sentences and the same words 
randomized in a string. Differences in retention are not 
observed. Various forms of mnemonic systems have been used to 
facilitate learning but there is consistent evidence showing that 
long-term memory is uninfluenced. 

Finally^ we should note that individual differences in rate 
of forgetting, if they exist at all, are very small in magnitude. 
A number of studies have shown that the rate at which a given 
list is learned by a group of subjects does not correlate with 
retention after tv;enty-four hours. This could mean that the 
variance added to forgetting by individual differences is mini- 
mal. However, the most convincing evidence on this matter comes 

from studies in which a group of subjects are given a constant 

number of trials to learn, with recall taken after twenty-four 
hours. Such studies show a high correlation between the number 
of correct responses. on the last learning trial and the number of 
items correctly recalled. These high correlations indicate that 
the level of learning achieved dominated the amount recalled; 
hence, individual differences in the rate of forgetting must have 
been minimal* 



Relevant Variables 

It i:3 much simpler to identify the independent variables, 
which do influence rates of forgetting, than those that do not. 
An obvious relevant variable, implied in the discussion just 
above, is the level or degree of learning achieved before the 
retention interval. The higher the degree of learning the better 
the long-term retention. It has long been assumed that 
overlearning led to better retention because oyerlearning simply 
v,as an extension of the degr ee-of-lear ning variable beyond the 
point at which all items had been first gotten correctly. 
However, this principle will probably need to be modified 
somewhat. According to unpublished work which I have heard 
about, if the task is a simple one overlearning will not 
facilitate retention; it will do so only if the task is a 
difficult one. 

Perhaps one of the most potent variable involved in 
retention is spoken of as spaced practice. Suppose, for example, 
that one group of subjects is given two learning trials a day for 
five days (spaced practice) while another group is given all 
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ten trials at a single sitting (massed practice). The recall of 
the group given spaced . practice will be far* better than that of 
that group given massed practice. These operations clearly have 
counterparts in the idea of cramming before an examination versus 
spaced study. If the test is given immediately after cramming^ 
there is no reason why performance should not be high. However^ 
it is the long-term retention measures that would be expected to 
differ^ with the spaced practice being far better than the 
massed. 

As a third variable I may mention a recent discovery which 
seems to indicate a further factor which influences long-term 
retention • If the svibject is given two or three tasks to learn 
simultaneously^ the retention of each list is better than if 
learned alone. In simultaneous learning the overall list of 
words consists of three distinguishable different classes of 
words which are simply^ randomized in the overall study list, but 
each class of items is tested separately. Not all possible 
artifacts have been ruled out of this paradigm but it now seems 
that the finding will hold up. We do not know just where this 
research will lead but the acquisition of several tasks 
simultaneously has ecological validity in that in school as well 
as in other situations we deal with several tasks more or less 
simultaneously. 

We come to the fourth variable that is known to influence 
the rate of forgetting of the simple case; this fourth variable 
is the learning of interfering tasks.. Both empirically and 
theoretically the forgetting produced for the target task by the 
learning of other tasks is of central importance. A large 
proportion of the work on long-term memory involves the study of 
the effect of interference from other tasks on the rate of 
forgetting, and innumerable studies in the past twenty years have 
examined the influence of interference in short-term memory 
paradigms . 

Finally, there is the fifth variable that influences 
long-term retention, namely the length of the retention interval. 
To summarize, the five variables which must be considered 
relevant variables for forgetting are level of learning, spaced 
practice, number of simultaneous tasks, interfering tasks, and 
length of the retention interval. 



Retroactive and Proactive Paradigms 

Retroactive inhibition has been studied for about 80 years 
To produce retroactive inhibition, a second task is inserted 
between the learning of the target list and its recall. Thus, 
the subject learns List T and 'then learns List X, before recall 
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of List*. T is requested. The control group would learn and recall 
only List T. The difference in the recall of the two groups 
defines retroactive inhibition. Proactive inhibition^ on the 
other hand^ has had a relatively short history with very little 
work being done on the phenomenon until after World War II. In 
proactive inhibition the X List is learned prior to learning the 
T List, with the control again not having learned the X List, It 
can be seen that in the case of proactive interference an 
interval must occur between the learning and retention of the T 
List before any interference could be expected. Thus, length of 
the retention interval is an important variable determining the 
magnitude of proactive inhibition. 

We must clearly understand the reasons for studying 
proactive and retroactive interference, and just how they may 
help us understand the causes of forgetting. The drosophila, or 
fruit fly, has been used ex tensively to study genetics. The 
major reason for this is that the fruit fly's life cycle is very 
brief and, therefore, biologists can study hereditary changes 
across many generations in a relatively short period of time. 
Nearly every discipline has its fruit fly; that is, nearly every 
discipline has developed ways of speeding up the study of its 
phenomena over the rate which occurs when nature simply runs her 
course. In the study of forgetting, retroactive and proactive 
inhibition are essentially our fruit flies. Many of us believe 
that interference is the major cause of forgetting. By the use 
of the two interference paradigms it is possible to speed up the 
processes which are assumed to occur in real life, and the 
magnitude of thef' interference phenomena can be amplified by these 
paradigms. 

The facts of retroactive and proactive inhibition, as 
suggested above, led us to believe that the simple forgetting 
discussed earlier must be due to interference from tasks learned 
outside the laboratory, and that the forgetting oi: tasks learned 
outside of the laboratory is due to interference from other tasks 
learned outside of the laboratory. Because so much forgetting 
can be produced in the laboratory by the retroactive and 
proactive paradigms, theorists have been led to the conclusion 
that all forgetting results from interference. The forgetting of 
any task in real life probably results from a double whammy; it 
may be interfered with both by tasks learned earlier and by tasks 
learned later. Whatever yoa learn from what you read today will, 
over time, be put upon other events you have learned or 
experienced and by those things that you will subsequently learn 
or experience. 



Mechanisms Underlying Inter fere nee 

Interference always involves some incompatibility among 
response tendencies. Research which has studied the magnitude 
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of the incompatibility has used various transfer paradigms^ all 
of which have' some representation in real life. Thus^ there is 
the A~Bf A-D paradigm in which paired-associate lists are used^ 
and in which the stimulus terms in the two lists are identical, 
with the response terms being different. The A-B, C-B paradigm, 
on the other hand, has the sam^^ response terms in the two lists 
but the stimulus terms differ. It may be noted that this 
paradigm is the prototype for concept learning. There are other 
transfer paradigms, but they need not be enumerated here. It is 
apparent that the amount of interference is in some way related 
to similarity of the materials, and up to a point the higher the 
similarity the greater the incompatibility, hence the greater the 
interference at the time of recall. 

A number of years ago it was suggested . that retroactive 
inhibition involves more than competition among incompatible 
response tendencies at the time of recall. More specifically, it 
was proposed that with many interference paradigms, such as A-B, 
A-D, there is an unlearning or weakening or extinction of the A-B 
association during the learning of A-D. The technique which was 
evolved for testing this idea required the subjects to try to 
give both response terms to the stimulus terms after both lists 
had been learned. The time allowed for recalling both response 
terms was essentially unlimited. The idea was that with unpaced 
testing there would be no competition between the two response 
tendencies so that if the first-list. responses could not be 
produced, it must be due to some factor other than competition., 
and thus was due to unlearning or extinction. Many experiments 
showed that something like unlearning must indeed have occurred. 
The subjects simply could not recall some of the first-list (A-B) 
response terms no matter how much time was given. This seemingly 
powerful analytical situation gradually began to disintegrate as 
studies began to show that the competition between the two 
response tendencies (A-B versus A-D) could occur even if there 
were no time pressures for the subject to respond. At the 
present time, therefore, it is doubtful that we should assume 
that a phenomenon such as unlearning is a part of the cause of 
the forgetting observed in the retroactive inhibition paradigm. 
Competition leading to interference remains as the only clear 
factor involved in forgetting. 

In a very general sense, .nterference is an inverse function 
of the discr iminabii ity between tasks. Contemporary studies are 
being directed at factors which vary the d iscr iminabii ity of the 
potentially interfering tasks. In some cases, for exampl.. , it 
appears that a temporal discrimination is involved. In fad, 
there is appreciable evid^^ce from both short-term memory studies 
and long-term memory studies that the loss of temporal discrim- 
ination is the primary cause of proactive interference. But this 
does not seem to be the case for retroactive inhibition. If the 
subject learns Task T, then Task X immediately, and then tries to 
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recall Task T, there seems to be little confusion as to which 
list the recalled items belong^ but still there is interference 
in that the subjects cannot recall some of the response terms. 
It was this fac-t which unlearning handled so nicely and we may 
eventually find that the concept of unlearning is still 
necessary. If so, it will require the invention of a test 
paradigm in which it is cleai* there is no competition. 



The Theoretical Gap 

The purpose f or study ing retroactive and proactive 
inhibition is that the outcomes of the experiments should direct 
us toward an understanding of the simple case of forgetting. 
However, there must be a theory which specifies how retroactive 
and proactive inhibition exhibit their influences in the simple 
case, and the theory must be capable of being tested. Such a 
theory had been proposed in great detail a number of years ago, 
with clear predictions flowing from the theory. Many tests have 
given little support to the theory. The basic idea that the 
simple case represents the forgetting produced by interference 
need not be discarded because of the failure of the one theory, 
A new transition theory is needed to close the gap between 
laboratory-produced interference and the case of simple forget- 
ting. If not forthcoming, our fruit fly will not have 
accomplished completely what it was intended to accomplish. 

It has sometimes been wondered how it is that we are able to 
carry on relatively organized and directed lives if interference 
is lurking around every corner ready to compete with each new 
thing we learn. There are a number of possible answers to this 
criticism. First, there is the possibility that in real life we 
do forget an enormous number of things v;e have learned. Try to 
remember in detail one's activities of the last ten hours, i?ry 
to remember many facts of history which at one time we knew 
"cold," Second^ there are variables (which were listed earlier) 
which seem to inoculate memories against interference. Third, 
there may be^ various organizational structures into which new 
learning ~ i's""placed" and these structures may be easily 
discriminable from each other, hence the new memories may be 
discriminated by the structure from potentially interfering 
memories. Nevertheless, within a structure some interference 
must be expected. In short, it does not seem that a satisfactory 
theory of forgetting is going to be written if interference is 
omitted as a factor. It may turn out that it is not the only 
factor, but surely it must be a factor. 
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